Linear Regression Final Project

Loading the data

The source of the data is here: https://archive.ics.uci.edu/ml/datasets/Wine+Quality
Original citation: P. Cortez, A. Cerdeira, F. Almeida, T. Matos and J. Reis. Modeling wine preferences by data mining from physicochemical properties. In Decision Support Systems, Elsevier, 47(4):547-553, 2009.

This data consists of two datasets. Each dataset contains quality ratings for wines as well as 11 features of these wines. The 12 variables are all continuous and numeric. The first dataset contains 1599 observations of red wines; the second dataset contains 4898 observations of white wines.

df <- as_tibble(read.table("winequality-red.csv", sep=";", header = TRUE))
summary(df$quality)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##   3.000   5.000   6.000   5.636   6.000   8.000

df_white <- as_tibble(read.table("winequality-white.csv", sep=";", header = TRUE))
summary(df_white$quality)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##   3.000   5.000   6.000   5.878   6.000   9.000

Cleaning the data

The variables in the data all have different scales. For example:

summary(df$fixed.acidity)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##    4.60    7.10    7.90    8.32    9.20   15.90

summary(df$sulphates)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##  0.3300  0.5500  0.6200  0.6581  0.7300  2.0000

The data will be standardized to make some plots of the data easier to visualize.

data_red <- as.data.frame(scale(df))
summary(data_red$fixed.acidity)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
## -2.1364 -0.7005 -0.2410  0.0000  0.5056  4.3538

data_white <- as.data.frame(scale(df_white))
summary(data_white$sulphates)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
## -2.3645 -0.6996 -0.1739  0.0000  0.5271  5.1711

Visualizing the data

Glancing at the raw data, we see that the red wine quality ratings are not distributed uniformly, and there are a few outlier ratings. Our model will probably not perform well when predicting red wine ratings below 4 or above 7.

hist(df$quality,
    main="Histogram of Red Wine Quality Ratings", 
    xlab="Rating", 
    las=1, 
    breaks=4)

The white wine ratings are distributed similarly, although a bit more symmetrically. Our model will probably not perform well when predicting white wine ratings below 4 or above 8.

hist(df_white$quality,
     main="Histogram of White Wine Quality Ratings", 
     xlab="Rating", 
     las=1, 
     breaks=5)

We may consider winsorizing or truncating our dataset to only consider observations where the rating was above 4 and below 7 (or 8, for white wine).

Further visualization

Looking at the standardized data, there are a few variables that immediately appear to be predictive of the quality. Alcohol is one:

ggplot(data_red, aes(x=alcohol, y=quality)) + 
  geom_point() +
  geom_smooth(method=lm) +
  labs(title="Alcohol Content vs. Quality Rating",
       subtitle="Red Wine",
       y="Quality",
       x="Alcohol")

Volatile acidity is another one:

ggplot(data_red, aes(x=volatile.acidity, y=quality)) + 
  geom_point() +
  geom_smooth(method=lm) +
  labs(title="Volatile Acidity vs. Quality Rating",
       subtitle="Red Wine",
       y="Quality",
       x="Acidity")

These variables will likely be useful for predicting quality ratings of white wine as well:

ggplot(data_white, aes(x=alcohol, y=quality)) + 
  geom_point() +
  geom_smooth(method=lm) +
  labs(title="Alcohol Content vs. Quality Rating",
       subtitle="White Wine",
       y="Quality",
       x="Alcohol")

ggplot(data_white, aes(x=volatile.acidity, y=quality)) + 
  geom_point() +
  geom_smooth(method=lm) +
  labs(title="Volatile Acidity vs. Quality Rating",
       subtitle="White Wine",
       y="Quality",
       x="Acidity")

Modeling

The data will be split into train and test datasets. The model will be trained on the train data and evaluated on the test data.

index = sample(1:nrow(df), 0.7*nrow(df)) 

train = df[index,] # Create the training data 
test = df[-index,] # Create the test data

dim(train)

## [1] 1119   12

dim(test)

## [1] 480  12

Rudimentary LM on whole dataset

lm_basic <- lm(data = df, quality ~ .)

summary(lm_basic)

## 
## Call:
## lm(formula = quality ~ ., data = df)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -2.68911 -0.36652 -0.04699  0.45202  2.02498 
## 
## Coefficients:
##                        Estimate Std. Error t value Pr(>|t|)    
## (Intercept)           2.197e+01  2.119e+01   1.036   0.3002    
## fixed.acidity         2.499e-02  2.595e-02   0.963   0.3357    
## volatile.acidity     -1.084e+00  1.211e-01  -8.948  < 2e-16 ***
## citric.acid          -1.826e-01  1.472e-01  -1.240   0.2150    
## residual.sugar        1.633e-02  1.500e-02   1.089   0.2765    
## chlorides            -1.874e+00  4.193e-01  -4.470 8.37e-06 ***
## free.sulfur.dioxide   4.361e-03  2.171e-03   2.009   0.0447 *  
## total.sulfur.dioxide -3.265e-03  7.287e-04  -4.480 8.00e-06 ***
## density              -1.788e+01  2.163e+01  -0.827   0.4086    
## pH                   -4.137e-01  1.916e-01  -2.159   0.0310 *  
## sulphates             9.163e-01  1.143e-01   8.014 2.13e-15 ***
## alcohol               2.762e-01  2.648e-02  10.429  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.648 on 1587 degrees of freedom
## Multiple R-squared:  0.3606, Adjusted R-squared:  0.3561 
## F-statistic: 81.35 on 11 and 1587 DF,  p-value: < 2.2e-16

Volatile acidity, chlorides, total sulfur dioxide, sulphates and alcohol are all very statistically significant, although our adjusted \(R^2\) is still quite low.

Ridge regression

train_control <- trainControl(method  = "cv", number = 5)

model_ridge <- train(quality ~ .,
                     data = df,
                     method = "ridge",           # method
                     trControl = train_control)        # cross validation

model_ridge

## Ridge Regression 
## 
## 1599 samples
##   11 predictor
## 
## No pre-processing
## Resampling: Cross-Validated (5 fold) 
## Summary of sample sizes: 1279, 1280, 1280, 1279, 1278 
## Resampling results across tuning parameters:
## 
##   lambda  RMSE       Rsquared   MAE      
##   0e+00   0.6508382  0.3530946  0.5040202
##   1e-04   0.6508355  0.3531007  0.5040170
##   1e-01   0.6504340  0.3546738  0.5027356
## 
## RMSE was used to select the optimal model using the smallest value.
## The final value used for the model was lambda = 0.1.

#something doesn't seem to be working here

model_stepwise <- train(quality ~ .,
                        data = df,
                        method = "glmStepAIC",
                        trControl = train_control)

## Start:  AIC=2474.52
## .outcome ~ fixed.acidity + volatile.acidity + citric.acid + residual.sugar + 
##     chlorides + free.sulfur.dioxide + total.sulfur.dioxide + 
##     density + pH + sulphates + alcohol
## 
##                        Df Deviance    AIC
## - residual.sugar        1   507.95 2472.6
## - density               1   508.01 2472.7
## - fixed.acidity         1   508.12 2473.0
## - citric.acid           1   508.37 2473.6
## <none>                      507.94 2474.5
## - pH                    1   508.93 2475.0
## - free.sulfur.dioxide   1   509.72 2477.0
## - chlorides             1   514.60 2489.2
## - total.sulfur.dioxide  1   516.99 2495.1
## - volatile.acidity      1   532.67 2533.3
## - sulphates             1   535.93 2541.1
## - alcohol               1   543.89 2560.0
## 
## Step:  AIC=2472.56
## .outcome ~ fixed.acidity + volatile.acidity + citric.acid + chlorides + 
##     free.sulfur.dioxide + total.sulfur.dioxide + density + pH + 
##     sulphates + alcohol
## 
##                        Df Deviance    AIC
## - density               1   508.01 2470.7
## - fixed.acidity         1   508.13 2471.0
## - citric.acid           1   508.38 2471.6
## <none>                      507.95 2472.6
## - pH                    1   509.14 2473.6
## - free.sulfur.dioxide   1   509.81 2475.2
## - chlorides             1   514.62 2487.2
## - total.sulfur.dioxide  1   517.02 2493.2
## - volatile.acidity      1   532.76 2531.5
## - sulphates             1   536.68 2540.9
## - alcohol               1   556.16 2586.5
## 
## Step:  AIC=2470.71
## .outcome ~ fixed.acidity + volatile.acidity + citric.acid + chlorides + 
##     free.sulfur.dioxide + total.sulfur.dioxide + pH + sulphates + 
##     alcohol
## 
##                        Df Deviance    AIC
## - fixed.acidity         1   508.13 2469.0
## - citric.acid           1   508.45 2469.8
## <none>                      508.01 2470.7
## - free.sulfur.dioxide   1   509.88 2473.4
## - pH                    1   509.93 2473.5
## - chlorides             1   514.86 2485.8
## - total.sulfur.dioxide  1   517.45 2492.2
## - volatile.acidity      1   533.56 2531.5
## - sulphates             1   537.01 2539.7
## - alcohol               1   593.10 2666.8
## 
## Step:  AIC=2469.01
## .outcome ~ volatile.acidity + citric.acid + chlorides + free.sulfur.dioxide + 
##     total.sulfur.dioxide + pH + sulphates + alcohol
## 
##                        Df Deviance    AIC
## - citric.acid           1   508.45 2467.8
## <none>                      508.13 2469.0
## - free.sulfur.dioxide   1   510.08 2471.9
## - pH                    1   511.61 2475.7
## - chlorides             1   515.85 2486.3
## - total.sulfur.dioxide  1   518.84 2493.7
## - volatile.acidity      1   534.32 2531.3
## - sulphates             1   537.60 2539.1
## - alcohol               1   593.56 2665.8
## 
## Step:  AIC=2467.82
## .outcome ~ volatile.acidity + chlorides + free.sulfur.dioxide + 
##     total.sulfur.dioxide + pH + sulphates + alcohol
## 
##                        Df Deviance    AIC
## <none>                      508.45 2467.8
## - free.sulfur.dioxide   1   510.68 2471.4
## - pH                    1   511.72 2474.0
## - chlorides             1   516.82 2486.7
## - total.sulfur.dioxide  1   520.04 2494.7
## - sulphates             1   537.62 2537.2
## - volatile.acidity      1   539.17 2540.8
## - alcohol               1   594.93 2666.7
## Start:  AIC=2547.9
## .outcome ~ fixed.acidity + volatile.acidity + citric.acid + residual.sugar + 
##     chlorides + free.sulfur.dioxide + total.sulfur.dioxide + 
##     density + pH + sulphates + alcohol
## 
##                        Df Deviance    AIC
## - density               1   538.63 2546.6
## - residual.sugar        1   538.94 2547.3
## - citric.acid           1   539.01 2547.5
## - fixed.acidity         1   539.04 2547.6
## <none>                      538.34 2547.9
## - pH                    1   539.27 2548.1
## - free.sulfur.dioxide   1   539.63 2549.0
## - chlorides             1   545.45 2562.7
## - total.sulfur.dioxide  1   545.79 2563.5
## - sulphates             1   557.36 2590.3
## - volatile.acidity      1   561.73 2600.2
## - alcohol               1   574.16 2628.2
## 
## Step:  AIC=2546.58
## .outcome ~ fixed.acidity + volatile.acidity + citric.acid + residual.sugar + 
##     chlorides + free.sulfur.dioxide + total.sulfur.dioxide + 
##     pH + sulphates + alcohol
## 
##                        Df Deviance    AIC
## - residual.sugar        1   538.95 2545.3
## - fixed.acidity         1   539.08 2545.7
## - citric.acid           1   539.31 2546.2
## <none>                      538.63 2546.6
## - free.sulfur.dioxide   1   540.02 2547.9
## - pH                    1   540.97 2550.1
## - chlorides             1   546.07 2562.1
## - total.sulfur.dioxide  1   546.29 2562.6
## - sulphates             1   557.67 2589.0
## - volatile.acidity      1   563.09 2601.3
## - alcohol               1   638.45 2761.9
## 
## Step:  AIC=2545.34
## .outcome ~ fixed.acidity + volatile.acidity + citric.acid + chlorides + 
##     free.sulfur.dioxide + total.sulfur.dioxide + pH + sulphates + 
##     alcohol
## 
##                        Df Deviance    AIC
## - fixed.acidity         1   539.47 2544.6
## - citric.acid           1   539.57 2544.8
## <none>                      538.95 2545.3
## - free.sulfur.dioxide   1   540.50 2547.0
## - pH                    1   541.25 2548.8
## - chlorides             1   546.20 2560.4
## - total.sulfur.dioxide  1   546.39 2560.9
## - sulphates             1   557.75 2587.2
## - volatile.acidity      1   563.14 2599.5
## - alcohol               1   640.92 2764.8
## 
## Step:  AIC=2544.58
## .outcome ~ volatile.acidity + citric.acid + chlorides + free.sulfur.dioxide + 
##     total.sulfur.dioxide + pH + sulphates + alcohol
## 
##                        Df Deviance    AIC
## - citric.acid           1   539.69 2543.1
## <none>                      539.47 2544.6
## - free.sulfur.dioxide   1   541.22 2546.7
## - pH                    1   544.60 2554.7
## - chlorides             1   548.29 2563.3
## - total.sulfur.dioxide  1   548.95 2564.8
## - sulphates             1   558.85 2587.7
## - volatile.acidity      1   563.39 2598.0
## - alcohol               1   641.46 2763.9
## 
## Step:  AIC=2543.1
## .outcome ~ volatile.acidity + chlorides + free.sulfur.dioxide + 
##     total.sulfur.dioxide + pH + sulphates + alcohol
## 
##                        Df Deviance    AIC
## <none>                      539.69 2543.1
## - free.sulfur.dioxide   1   541.65 2545.7
## - pH                    1   545.13 2553.9
## - chlorides             1   549.03 2563.0
## - total.sulfur.dioxide  1   549.91 2565.1
## - sulphates             1   558.87 2585.7
## - volatile.acidity      1   568.21 2606.9
## - alcohol               1   643.75 2766.4
## Start:  AIC=2532.13
## .outcome ~ fixed.acidity + volatile.acidity + citric.acid + residual.sugar + 
##     chlorides + free.sulfur.dioxide + total.sulfur.dioxide + 
##     density + pH + sulphates + alcohol
## 
##                        Df Deviance    AIC
## - free.sulfur.dioxide   1   531.19 2530.7
## - fixed.acidity         1   531.37 2531.2
## - density               1   531.49 2531.4
## <none>                      530.94 2532.1
## - citric.acid           1   531.80 2532.2
## - pH                    1   532.10 2532.9
## - residual.sugar        1   532.66 2534.3
## - total.sulfur.dioxide  1   535.66 2541.5
## - chlorides             1   537.41 2545.6
## - sulphates             1   552.84 2581.9
## - volatile.acidity      1   562.91 2605.0
## - alcohol               1   563.50 2606.3
## 
## Step:  AIC=2530.73
## .outcome ~ fixed.acidity + volatile.acidity + citric.acid + residual.sugar + 
##     chlorides + total.sulfur.dioxide + density + pH + sulphates + 
##     alcohol
## 
##                        Df Deviance    AIC
## - fixed.acidity         1   531.69 2529.9
## - density               1   531.81 2530.2
## <none>                      531.19 2530.7
## - pH                    1   532.22 2531.2
## - citric.acid           1   532.23 2531.2
## - residual.sugar        1   533.16 2533.5
## - total.sulfur.dioxide  1   537.22 2543.2
## - chlorides             1   537.53 2543.9
## - sulphates             1   553.39 2581.2
## - alcohol               1   563.58 2604.5
## - volatile.acidity      1   565.08 2607.9
## 
## Step:  AIC=2529.94
## .outcome ~ volatile.acidity + citric.acid + residual.sugar + 
##     chlorides + total.sulfur.dioxide + density + pH + sulphates + 
##     alcohol
## 
##                        Df Deviance    AIC
## - density               1   531.83 2528.3
## - citric.acid           1   532.40 2529.6
## <none>                      531.69 2529.9
## - residual.sugar        1   533.18 2531.5
## - pH                    1   536.56 2539.6
## - total.sulfur.dioxide  1   538.75 2544.8
## - chlorides             1   539.26 2546.0
## - sulphates             1   553.40 2579.2
## - volatile.acidity      1   565.36 2606.5
## - alcohol               1   586.54 2653.6
## 
## Step:  AIC=2528.27
## .outcome ~ volatile.acidity + citric.acid + residual.sugar + 
##     chlorides + total.sulfur.dioxide + pH + sulphates + alcohol
## 
##                        Df Deviance    AIC
## <none>                      531.83 2528.3
## - citric.acid           1   533.12 2529.4
## - residual.sugar        1   533.24 2529.7
## - pH                    1   536.72 2538.0
## - total.sulfur.dioxide  1   538.81 2542.9
## - chlorides             1   539.26 2544.0
## - sulphates             1   553.68 2577.8
## - volatile.acidity      1   569.31 2613.4
## - alcohol               1   628.85 2740.8
## Start:  AIC=2552.28
## .outcome ~ fixed.acidity + volatile.acidity + citric.acid + residual.sugar + 
##     chlorides + free.sulfur.dioxide + total.sulfur.dioxide + 
##     density + pH + sulphates + alcohol
## 
##                        Df Deviance    AIC
## - residual.sugar        1   539.37 2550.3
## - fixed.acidity         1   539.39 2550.3
## - density               1   539.41 2550.4
## - citric.acid           1   539.63 2550.9
## <none>                      539.37 2552.3
## - free.sulfur.dioxide   1   541.75 2555.9
## - pH                    1   543.22 2559.4
## - total.sulfur.dioxide  1   546.04 2566.0
## - chlorides             1   547.69 2569.9
## - sulphates             1   560.91 2600.4
## - volatile.acidity      1   568.67 2618.0
## - alcohol               1   577.06 2636.7
## 
## Step:  AIC=2550.29
## .outcome ~ fixed.acidity + volatile.acidity + citric.acid + chlorides + 
##     free.sulfur.dioxide + total.sulfur.dioxide + density + pH + 
##     sulphates + alcohol
## 
##                        Df Deviance    AIC
## - fixed.acidity         1   539.39 2548.3
## - density               1   539.41 2548.4
## - citric.acid           1   539.63 2548.9
## <none>                      539.37 2550.3
## - free.sulfur.dioxide   1   541.81 2554.1
## - pH                    1   543.83 2558.8
## - total.sulfur.dioxide  1   546.04 2564.0
## - chlorides             1   547.72 2567.9
## - sulphates             1   561.75 2600.3
## - volatile.acidity      1   568.69 2616.1
## - alcohol               1   590.12 2663.4
## 
## Step:  AIC=2548.33
## .outcome ~ volatile.acidity + citric.acid + chlorides + free.sulfur.dioxide + 
##     total.sulfur.dioxide + density + pH + sulphates + alcohol
## 
##                        Df Deviance    AIC
## - density               1   539.41 2546.4
## - citric.acid           1   539.64 2546.9
## <none>                      539.39 2548.3
## - free.sulfur.dioxide   1   541.87 2552.2
## - total.sulfur.dioxide  1   546.80 2563.8
## - pH                    1   548.11 2566.9
## - chlorides             1   548.50 2567.8
## - sulphates             1   561.81 2598.5
## - volatile.acidity      1   568.75 2614.2
## - alcohol               1   601.67 2686.2
## 
## Step:  AIC=2546.39
## .outcome ~ volatile.acidity + citric.acid + chlorides + free.sulfur.dioxide + 
##     total.sulfur.dioxide + pH + sulphates + alcohol
## 
##                        Df Deviance    AIC
## - citric.acid           1   539.85 2545.4
## <none>                      539.41 2546.4
## - free.sulfur.dioxide   1   541.88 2550.2
## - total.sulfur.dioxide  1   546.81 2561.8
## - pH                    1   548.18 2565.0
## - chlorides             1   548.59 2566.0
## - sulphates             1   562.37 2597.7
## - volatile.acidity      1   572.50 2620.6
## - alcohol               1   635.71 2754.6
## 
## Step:  AIC=2545.43
## .outcome ~ volatile.acidity + chlorides + free.sulfur.dioxide + 
##     total.sulfur.dioxide + pH + sulphates + alcohol
## 
##                        Df Deviance    AIC
## <none>                      539.85 2545.4
## - free.sulfur.dioxide   1   542.63 2550.0
## - total.sulfur.dioxide  1   547.87 2562.3
## - pH                    1   548.84 2564.6
## - chlorides             1   549.97 2567.2
## - sulphates             1   562.43 2595.9
## - volatile.acidity      1   579.40 2633.9
## - alcohol               1   637.20 2755.6
## Start:  AIC=2558.93
## .outcome ~ fixed.acidity + volatile.acidity + citric.acid + residual.sugar + 
##     chlorides + free.sulfur.dioxide + total.sulfur.dioxide + 
##     density + pH + sulphates + alcohol
## 
##                        Df Deviance    AIC
## - density               1   542.99 2557.9
## - fixed.acidity         1   543.07 2558.1
## - citric.acid           1   543.08 2558.1
## - residual.sugar        1   543.42 2558.9
## <none>                      542.59 2558.9
## - pH                    1   544.11 2560.5
## - free.sulfur.dioxide   1   544.39 2561.2
## - chlorides             1   547.76 2569.1
## - total.sulfur.dioxide  1   549.16 2572.3
## - sulphates             1   560.96 2599.5
## - volatile.acidity      1   567.47 2614.3
## - alcohol               1   582.23 2647.1
## 
## Step:  AIC=2557.86
## .outcome ~ fixed.acidity + volatile.acidity + citric.acid + residual.sugar + 
##     chlorides + free.sulfur.dioxide + total.sulfur.dioxide + 
##     pH + sulphates + alcohol
## 
##                        Df Deviance    AIC
## - fixed.acidity         1   543.09 2556.1
## - residual.sugar        1   543.43 2556.9
## - citric.acid           1   543.48 2557.0
## <none>                      542.99 2557.9
## - free.sulfur.dioxide   1   544.93 2560.4
## - pH                    1   546.84 2564.9
## - chlorides             1   548.55 2568.9
## - total.sulfur.dioxide  1   549.81 2571.8
## - sulphates             1   561.19 2598.0
## - volatile.acidity      1   569.06 2615.8
## - alcohol               1   645.62 2777.3
## 
## Step:  AIC=2556.1
## .outcome ~ volatile.acidity + citric.acid + residual.sugar + 
##     chlorides + free.sulfur.dioxide + total.sulfur.dioxide + 
##     pH + sulphates + alcohol
## 
##                        Df Deviance    AIC
## - citric.acid           1   543.49 2555.0
## - residual.sugar        1   543.56 2555.2
## <none>                      543.09 2556.1
## - free.sulfur.dioxide   1   545.10 2558.8
## - pH                    1   549.30 2568.7
## - chlorides             1   549.48 2569.1
## - total.sulfur.dioxide  1   550.99 2572.6
## - sulphates             1   561.51 2596.8
## - volatile.acidity      1   570.08 2616.1
## - alcohol               1   646.63 2777.3
## 
## Step:  AIC=2555.04
## .outcome ~ volatile.acidity + residual.sugar + chlorides + free.sulfur.dioxide + 
##     total.sulfur.dioxide + pH + sulphates + alcohol
## 
##                        Df Deviance    AIC
## - residual.sugar        1   543.86 2553.9
## <none>                      543.49 2555.0
## - free.sulfur.dioxide   1   545.83 2558.5
## - pH                    1   549.69 2567.6
## - chlorides             1   550.66 2569.8
## - total.sulfur.dioxide  1   552.06 2573.1
## - sulphates             1   561.66 2595.1
## - volatile.acidity      1   575.28 2625.8
## - alcohol               1   648.47 2778.9
## 
## Step:  AIC=2553.92
## .outcome ~ volatile.acidity + chlorides + free.sulfur.dioxide + 
##     total.sulfur.dioxide + pH + sulphates + alcohol
## 
##                        Df Deviance    AIC
## <none>                      543.86 2553.9
## - free.sulfur.dioxide   1   546.31 2557.7
## - pH                    1   550.51 2567.5
## - chlorides             1   550.85 2568.3
## - total.sulfur.dioxide  1   552.12 2571.2
## - sulphates             1   561.80 2593.4
## - volatile.acidity      1   575.60 2624.5
## - alcohol               1   651.50 2782.9
## Start:  AIC=3164.28
## .outcome ~ fixed.acidity + volatile.acidity + citric.acid + residual.sugar + 
##     chlorides + free.sulfur.dioxide + total.sulfur.dioxide + 
##     density + pH + sulphates + alcohol
## 
##                        Df Deviance    AIC
## - density               1   666.70 3163.0
## - fixed.acidity         1   666.80 3163.2
## - residual.sugar        1   666.91 3163.5
## - citric.acid           1   667.06 3163.8
## <none>                      666.41 3164.3
## - free.sulfur.dioxide   1   668.10 3166.3
## - pH                    1   668.37 3167.0
## - chlorides             1   674.80 3182.3
## - total.sulfur.dioxide  1   674.84 3182.4
## - sulphates             1   693.38 3225.7
## - volatile.acidity      1   700.03 3241.0
## - alcohol               1   712.08 3268.3
## 
## Step:  AIC=3162.96
## .outcome ~ fixed.acidity + volatile.acidity + citric.acid + residual.sugar + 
##     chlorides + free.sulfur.dioxide + total.sulfur.dioxide + 
##     pH + sulphates + alcohol
## 
##                        Df Deviance    AIC
## - fixed.acidity         1   666.81 3161.2
## - residual.sugar        1   666.93 3161.5
## - citric.acid           1   667.35 3162.5
## <none>                      666.70 3163.0
## - free.sulfur.dioxide   1   668.53 3165.3
## - pH                    1   671.02 3171.3
## - total.sulfur.dioxide  1   675.43 3181.8
## - chlorides             1   675.46 3181.8
## - sulphates             1   693.98 3225.1
## - volatile.acidity      1   701.70 3242.8
## - alcohol               1   786.37 3424.9
## 
## Step:  AIC=3161.22
## .outcome ~ volatile.acidity + citric.acid + residual.sugar + 
##     chlorides + free.sulfur.dioxide + total.sulfur.dioxide + 
##     pH + sulphates + alcohol
## 
##                        Df Deviance    AIC
## - residual.sugar        1   667.06 3159.8
## - citric.acid           1   667.37 3160.6
## <none>                      666.81 3161.2
## - free.sulfur.dioxide   1   668.71 3163.8
## - pH                    1   673.87 3176.1
## - chlorides             1   676.75 3182.9
## - total.sulfur.dioxide  1   676.84 3183.1
## - sulphates             1   694.48 3224.2
## - volatile.acidity      1   703.04 3243.8
## - alcohol               1   787.44 3425.1
## 
## Step:  AIC=3159.84
## .outcome ~ volatile.acidity + citric.acid + chlorides + free.sulfur.dioxide + 
##     total.sulfur.dioxide + pH + sulphates + alcohol
## 
##                        Df Deviance    AIC
## - citric.acid           1   667.54 3159.0
## <none>                      667.06 3159.8
## - free.sulfur.dioxide   1   669.13 3162.8
## - pH                    1   674.20 3174.9
## - total.sulfur.dioxide  1   676.89 3181.2
## - chlorides             1   676.89 3181.2
## - sulphates             1   694.51 3222.3
## - volatile.acidity      1   703.04 3241.8
## - alcohol               1   789.73 3427.8
## 
## Step:  AIC=3158.98
## .outcome ~ volatile.acidity + chlorides + free.sulfur.dioxide + 
##     total.sulfur.dioxide + pH + sulphates + alcohol
## 
##                        Df Deviance    AIC
## <none>                      667.54 3159.0
## - free.sulfur.dioxide   1   669.93 3162.7
## - pH                    1   674.61 3173.8
## - total.sulfur.dioxide  1   678.32 3182.6
## - chlorides             1   678.35 3182.7
## - sulphates             1   694.60 3220.5
## - volatile.acidity      1   709.85 3255.3
## - alcohol               1   792.02 3430.4

#model_stepwise

Explore anything highly correlated:

correlation_df <- cor(df)

#correlation_df

# fixed acidity to ph and sulphates

correlation_df_melt <- melt(correlation_df)

gz <- ggplot(correlation_df_melt, mapping = aes(x = Var1, y = Var2, fill = value)) + 
  geom_tile() + 
  theme(axis.text.x = element_text(angle = 90, hjust = 1))+
  theme(text = element_text(size = 8)) + 
  ggtitle("Heat map for correlation") + 
  ylab("")+
  xlab("")+
  scale_fill_distiller(palette = "RdPu")

ggplotly(gz, tooltip = "text")

minimized_lm_df <- df %>% 
  select(-free.sulfur.dioxide, -citric.acid, -density, -volatile.acidity, -pH)

minimized_lm <- lm(data = minimized_lm_df, quality ~ .)

summary(minimized_lm)

## 
## Call:
## lm(formula = quality ~ ., data = minimized_lm_df)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -2.71086 -0.37485 -0.06023  0.48038  2.08242 
## 
## Coefficients:
##                        Estimate Std. Error t value Pr(>|t|)    
## (Intercept)           1.5204529  0.2113680   7.193 9.69e-13 ***
## fixed.acidity         0.0485196  0.0101132   4.798 1.76e-06 ***
## residual.sugar        0.0057865  0.0124562   0.465    0.642    
## chlorides            -2.3443435  0.4030377  -5.817 7.24e-09 ***
## total.sulfur.dioxide -0.0023418  0.0005467  -4.283 1.95e-05 ***
## sulphates             1.1850119  0.1116567  10.613  < 2e-16 ***
## alcohol               0.3100041  0.0171438  18.083  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.6742 on 1592 degrees of freedom
## Multiple R-squared:  0.3057, Adjusted R-squared:  0.3031 
## F-statistic: 116.8 on 6 and 1592 DF,  p-value: < 2.2e-16

correlations

findCorrelation(
  cor(df),
  cutoff = 0.5,
  verbose = TRUE,
  names = TRUE
)

## Compare row 3  and column  1 with corr  0.672 
##   Means:  0.293 vs 0.19 so flagging column 3 
## Compare row 1  and column  8 with corr  0.668 
##   Means:  0.245 vs 0.172 so flagging column 1 
## Compare row 7  and column  6 with corr  0.668 
##   Means:  0.174 vs 0.169 so flagging column 7 
## All correlations <= 0.5

## [1] "citric.acid"          "fixed.acidity"        "total.sulfur.dioxide"

ggcorrplot(correlation_df, hc.order = TRUE, type = "lower")

df_less_acid <- df %>% 
  select(-citric.acid, -fixed.acidity)

lm_less_acid <- lm(data = df_less_acid, quality ~ .)
summary(lm_less_acid)

## 
## Call:
## lm(formula = quality ~ ., data = df_less_acid)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -2.66740 -0.37223 -0.04556  0.46264  2.03092 
## 
## Coefficients:
##                        Estimate Std. Error t value Pr(>|t|)    
## (Intercept)          12.0575013 12.0089257   1.004   0.3155    
## volatile.acidity     -1.0127925  0.1009279 -10.035  < 2e-16 ***
## residual.sugar        0.0115046  0.0134606   0.855   0.3929    
## chlorides            -2.0490938  0.3992190  -5.133 3.21e-07 ***
## free.sulfur.dioxide   0.0048741  0.0021385   2.279   0.0228 *  
## total.sulfur.dioxide -0.0035687  0.0006939  -5.143 3.03e-07 ***
## density              -7.5668358 11.8648390  -0.638   0.5237    
## pH                   -0.4919908  0.1210931  -4.063 5.08e-05 ***
## sulphates             0.9024434  0.1129657   7.989 2.60e-15 ***
## alcohol               0.2810022  0.0201992  13.912  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.648 on 1589 degrees of freedom
## Multiple R-squared:  0.3598, Adjusted R-squared:  0.3562 
## F-statistic: 99.22 on 9 and 1589 DF,  p-value: < 2.2e-16

Lasso

plot(df)

Full scaled analysis

colnames(df)

##  [1] "fixed.acidity"        "volatile.acidity"     "citric.acid"         
##  [4] "residual.sugar"       "chlorides"            "free.sulfur.dioxide" 
##  [7] "total.sulfur.dioxide" "density"              "pH"                  
## [10] "sulphates"            "alcohol"              "quality"

cols = c('fixed.acidity', 'volatile.acidity', 'citric.acid', 'residual.sugar', 'chlorides', 'free.sulfur.dioxide', 'total.sulfur.dioxide', 'density', 'pH', 'sulphates', 'alcohol')

pre_proc_val <- preProcess(train[,cols], method = c("center", "scale"))

train[,cols] = predict(pre_proc_val, train[,cols])
test[,cols] = predict(pre_proc_val, test[,cols])

summary(train)

##  fixed.acidity     volatile.acidity    citric.acid       residual.sugar    
##  Min.   :-2.1214   Min.   :-2.29403   Min.   :-1.38396   Min.   :-1.19082  
##  1st Qu.:-0.7040   1st Qu.:-0.72284   1st Qu.:-0.92625   1st Qu.:-0.46111  
##  Median :-0.2504   Median :-0.04947   Median :-0.06167   Median :-0.24219  
##  Mean   : 0.0000   Mean   : 0.00000   Mean   : 0.00000   Mean   : 0.00000  
##  3rd Qu.: 0.5434   3rd Qu.: 0.62390   3rd Qu.: 0.80290   3rd Qu.: 0.04969  
##  Max.   : 4.1152   Max.   : 4.49577   Max.   : 3.70176   Max.   : 9.46298  
##    chlorides        free.sulfur.dioxide total.sulfur.dioxide    density        
##  Min.   :-1.60030   Min.   :-1.3925     Min.   :-1.2082      Min.   :-3.59039  
##  1st Qu.:-0.36897   1st Qu.:-0.8179     1st Qu.:-0.7438      1st Qu.:-0.62763  
##  Median :-0.17790   Median :-0.2434     Median :-0.2794      Median : 0.01528  
##  Mean   : 0.00000   Mean   : 0.0000     Mean   : 0.0000      Mean   : 0.00000  
##  3rd Qu.: 0.05563   3rd Qu.: 0.5227     3rd Qu.: 0.4636      3rd Qu.: 0.55640  
##  Max.   :11.11635   Max.   : 5.0235     Max.   : 7.2123      Max.   : 3.41736  
##        pH             sulphates          alcohol           quality     
##  Min.   :-3.65206   Min.   :-1.9916   Min.   :-1.9179   Min.   :3.000  
##  1st Qu.:-0.66269   1st Qu.:-0.6066   1st Qu.:-0.8757   1st Qu.:5.000  
##  Median :-0.02666   Median :-0.2452   Median :-0.2125   Median :6.000  
##  Mean   : 0.00000   Mean   : 0.0000   Mean   : 0.0000   Mean   :5.626  
##  3rd Qu.: 0.54577   3rd Qu.: 0.4172   3rd Qu.: 0.6402   3rd Qu.:6.000  
##  Max.   : 4.42559   Max.   : 8.0652   Max.   : 3.3879   Max.   :8.000

#generic LM

lm_2 <- lm(quality ~ ., data = train)
summary(lm_2)

## 
## Call:
## lm(formula = quality ~ ., data = train)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -2.69037 -0.36050 -0.04272  0.46333  2.09045 
## 
## Coefficients:
##                       Estimate Std. Error t value Pr(>|t|)    
## (Intercept)           5.625559   0.019101 294.523  < 2e-16 ***
## fixed.acidity         0.016831   0.053411   0.315 0.752733    
## volatile.acidity     -0.220902   0.026177  -8.439  < 2e-16 ***
## citric.acid          -0.068388   0.034985  -1.955 0.050861 .  
## residual.sugar       -0.008401   0.024615  -0.341 0.732932    
## chlorides            -0.095642   0.023189  -4.124 4.00e-05 ***
## free.sulfur.dioxide   0.057265   0.026656   2.148 0.031910 *  
## total.sulfur.dioxide -0.095623   0.028346  -3.373 0.000768 ***
## density               0.025079   0.047489   0.528 0.597530    
## pH                   -0.095685   0.035111  -2.725 0.006527 ** 
## sulphates             0.151876   0.022769   6.670 4.02e-11 ***
## alcohol               0.318605   0.032827   9.706  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.6389 on 1107 degrees of freedom
## Multiple R-squared:  0.3671, Adjusted R-squared:  0.3609 
## F-statistic: 58.38 on 11 and 1107 DF,  p-value: < 2.2e-16

Regularization

cols_reg = c('fixed.acidity', 'volatile.acidity', 'citric.acid', 'residual.sugar', 'chlorides', 'free.sulfur.dioxide', 'total.sulfur.dioxide', 'density', 'pH', 'sulphates', 'alcohol', 'quality')

dummies <- dummyVars(quality ~ ., data = df[,cols_reg])

train_dummies = predict(dummies, newdata = train[,cols_reg])

test_dummies = predict(dummies, newdata = test[,cols_reg])

print(dim(train_dummies)); print(dim(test_dummies))

## [1] 1119   11

## [1] 480  11

Ridge Regression

x = as.matrix(train_dummies)
y_train = train$quality

x_test = as.matrix(test_dummies)
y_test = test$quality

lambdas <- 10^seq(2, -3, by = -.1)
ridge_reg = glmnet(x, y_train, nlambda = 25, alpha = 0, family = 'gaussian', lambda = lambdas)

summary(ridge_reg)

##           Length Class     Mode   
## a0         51    -none-    numeric
## beta      561    dgCMatrix S4     
## df         51    -none-    numeric
## dim         2    -none-    numeric
## lambda     51    -none-    numeric
## dev.ratio  51    -none-    numeric
## nulldev     1    -none-    numeric
## npasses     1    -none-    numeric
## jerr        1    -none-    numeric
## offset      1    -none-    logical
## call        7    -none-    call   
## nobs        1    -none-    numeric

Optimal Lambda

cv_ridge <- cv.glmnet(x, y_train, alpha = 0, lambda = lambdas)
optimal_lambda <- cv_ridge$lambda.min
optimal_lambda

## [1] 0.01995262

Formal metrics for Ridge

# Compute R^2 from true and predicted values
eval_results <- function(true, predicted, df) {
  SSE <- sum((predicted - true)^2)
  SST <- sum((true - mean(true))^2)
  R_square <- 1 - SSE / SST
  RMSE = sqrt(SSE/nrow(df))

  
  # Model performance metrics
data.frame(
  RMSE = RMSE,
  Rsquare = R_square
)
  
}

# Prediction and evaluation on train data
predictions_train <- predict(ridge_reg, s = optimal_lambda, newx = x)
eval_results(y_train, predictions_train, train)

##        RMSE   Rsquare
## 1 0.6357245 0.3667089

# Prediction and evaluation on test data
predictions_test <- predict(ridge_reg, s = optimal_lambda, newx = x_test)
eval_results(y_test, predictions_test, test)

##        RMSE   Rsquare
## 1 0.6710556 0.3402934

Lasso

lambdas <- 10^seq(2, -3, by = -.1)

# Setting alpha = 1 implements lasso regression
lasso_reg <- cv.glmnet(x, y_train, alpha = 1, lambda = lambdas, standardize = TRUE, nfolds = 5)

# Best 
lambda_best <- lasso_reg$lambda.min 
lambda_best

## [1] 0.006309573

lasso_model <- glmnet(x, y_train, alpha = 1, lambda = lambda_best, standardize = TRUE)

predictions_train <- predict(lasso_model, s = lambda_best, newx = x)
eval_results(y_train, predictions_train, train)

##        RMSE   Rsquare
## 1 0.6362121 0.3657369

predictions_test <- predict(lasso_model, s = lambda_best, newx = x_test)
eval_results(y_test, predictions_test, test)

##        RMSE  Rsquare
## 1 0.6696633 0.343028

Mix of ridge and lasso (elasticnet)

# Set training control
train_cont <- trainControl(method = "repeatedcv",
                              number = 10,
                              repeats = 5,
                              search = "random",
                              verboseIter = TRUE)

# Train the model
elastic_reg <- train(quality ~ .,
                           data = train,
                           method = "glmnet",
                           preProcess = c("center", "scale"),
                           tuneLength = 10,
                           trControl = train_cont)

## + Fold01.Rep1: alpha=0.30285, lambda=0.004666 
## - Fold01.Rep1: alpha=0.30285, lambda=0.004666 
## + Fold01.Rep1: alpha=0.02355, lambda=0.061703 
## - Fold01.Rep1: alpha=0.02355, lambda=0.061703 
## + Fold01.Rep1: alpha=0.32050, lambda=0.005575 
## - Fold01.Rep1: alpha=0.32050, lambda=0.005575 
## + Fold01.Rep1: alpha=0.03179, lambda=0.019563 
## - Fold01.Rep1: alpha=0.03179, lambda=0.019563 
## + Fold01.Rep1: alpha=0.55008, lambda=0.600447 
## - Fold01.Rep1: alpha=0.55008, lambda=0.600447 
## + Fold01.Rep1: alpha=0.71987, lambda=0.017336 
## - Fold01.Rep1: alpha=0.71987, lambda=0.017336 
## + Fold01.Rep1: alpha=0.16961, lambda=0.086625 
## - Fold01.Rep1: alpha=0.16961, lambda=0.086625 
## + Fold01.Rep1: alpha=0.61948, lambda=0.011854 
## - Fold01.Rep1: alpha=0.61948, lambda=0.011854 
## + Fold01.Rep1: alpha=0.18579, lambda=0.002510 
## - Fold01.Rep1: alpha=0.18579, lambda=0.002510 
## + Fold01.Rep1: alpha=0.36092, lambda=0.018533 
## - Fold01.Rep1: alpha=0.36092, lambda=0.018533 
## + Fold02.Rep1: alpha=0.30285, lambda=0.004666 
## - Fold02.Rep1: alpha=0.30285, lambda=0.004666 
## + Fold02.Rep1: alpha=0.02355, lambda=0.061703 
## - Fold02.Rep1: alpha=0.02355, lambda=0.061703 
## + Fold02.Rep1: alpha=0.32050, lambda=0.005575 
## - Fold02.Rep1: alpha=0.32050, lambda=0.005575 
## + Fold02.Rep1: alpha=0.03179, lambda=0.019563 
## - Fold02.Rep1: alpha=0.03179, lambda=0.019563 
## + Fold02.Rep1: alpha=0.55008, lambda=0.600447 
## - Fold02.Rep1: alpha=0.55008, lambda=0.600447 
## + Fold02.Rep1: alpha=0.71987, lambda=0.017336 
## - Fold02.Rep1: alpha=0.71987, lambda=0.017336 
## + Fold02.Rep1: alpha=0.16961, lambda=0.086625 
## - Fold02.Rep1: alpha=0.16961, lambda=0.086625 
## + Fold02.Rep1: alpha=0.61948, lambda=0.011854 
## - Fold02.Rep1: alpha=0.61948, lambda=0.011854 
## + Fold02.Rep1: alpha=0.18579, lambda=0.002510 
## - Fold02.Rep1: alpha=0.18579, lambda=0.002510 
## + Fold02.Rep1: alpha=0.36092, lambda=0.018533 
## - Fold02.Rep1: alpha=0.36092, lambda=0.018533 
## + Fold03.Rep1: alpha=0.30285, lambda=0.004666 
## - Fold03.Rep1: alpha=0.30285, lambda=0.004666 
## + Fold03.Rep1: alpha=0.02355, lambda=0.061703 
## - Fold03.Rep1: alpha=0.02355, lambda=0.061703 
## + Fold03.Rep1: alpha=0.32050, lambda=0.005575 
## - Fold03.Rep1: alpha=0.32050, lambda=0.005575 
## + Fold03.Rep1: alpha=0.03179, lambda=0.019563 
## - Fold03.Rep1: alpha=0.03179, lambda=0.019563 
## + Fold03.Rep1: alpha=0.55008, lambda=0.600447 
## - Fold03.Rep1: alpha=0.55008, lambda=0.600447 
## + Fold03.Rep1: alpha=0.71987, lambda=0.017336 
## - Fold03.Rep1: alpha=0.71987, lambda=0.017336 
## + Fold03.Rep1: alpha=0.16961, lambda=0.086625 
## - Fold03.Rep1: alpha=0.16961, lambda=0.086625 
## + Fold03.Rep1: alpha=0.61948, lambda=0.011854 
## - Fold03.Rep1: alpha=0.61948, lambda=0.011854 
## + Fold03.Rep1: alpha=0.18579, lambda=0.002510 
## - Fold03.Rep1: alpha=0.18579, lambda=0.002510 
## + Fold03.Rep1: alpha=0.36092, lambda=0.018533 
## - Fold03.Rep1: alpha=0.36092, lambda=0.018533 
## + Fold04.Rep1: alpha=0.30285, lambda=0.004666 
## - Fold04.Rep1: alpha=0.30285, lambda=0.004666 
## + Fold04.Rep1: alpha=0.02355, lambda=0.061703 
## - Fold04.Rep1: alpha=0.02355, lambda=0.061703 
## + Fold04.Rep1: alpha=0.32050, lambda=0.005575 
## - Fold04.Rep1: alpha=0.32050, lambda=0.005575 
## + Fold04.Rep1: alpha=0.03179, lambda=0.019563 
## - Fold04.Rep1: alpha=0.03179, lambda=0.019563 
## + Fold04.Rep1: alpha=0.55008, lambda=0.600447 
## - Fold04.Rep1: alpha=0.55008, lambda=0.600447 
## + Fold04.Rep1: alpha=0.71987, lambda=0.017336 
## - Fold04.Rep1: alpha=0.71987, lambda=0.017336 
## + Fold04.Rep1: alpha=0.16961, lambda=0.086625 
## - Fold04.Rep1: alpha=0.16961, lambda=0.086625 
## + Fold04.Rep1: alpha=0.61948, lambda=0.011854 
## - Fold04.Rep1: alpha=0.61948, lambda=0.011854 
## + Fold04.Rep1: alpha=0.18579, lambda=0.002510 
## - Fold04.Rep1: alpha=0.18579, lambda=0.002510 
## + Fold04.Rep1: alpha=0.36092, lambda=0.018533 
## - Fold04.Rep1: alpha=0.36092, lambda=0.018533 
## + Fold05.Rep1: alpha=0.30285, lambda=0.004666 
## - Fold05.Rep1: alpha=0.30285, lambda=0.004666 
## + Fold05.Rep1: alpha=0.02355, lambda=0.061703 
## - Fold05.Rep1: alpha=0.02355, lambda=0.061703 
## + Fold05.Rep1: alpha=0.32050, lambda=0.005575 
## - Fold05.Rep1: alpha=0.32050, lambda=0.005575 
## + Fold05.Rep1: alpha=0.03179, lambda=0.019563 
## - Fold05.Rep1: alpha=0.03179, lambda=0.019563 
## + Fold05.Rep1: alpha=0.55008, lambda=0.600447 
## - Fold05.Rep1: alpha=0.55008, lambda=0.600447 
## + Fold05.Rep1: alpha=0.71987, lambda=0.017336 
## - Fold05.Rep1: alpha=0.71987, lambda=0.017336 
## + Fold05.Rep1: alpha=0.16961, lambda=0.086625 
## - Fold05.Rep1: alpha=0.16961, lambda=0.086625 
## + Fold05.Rep1: alpha=0.61948, lambda=0.011854 
## - Fold05.Rep1: alpha=0.61948, lambda=0.011854 
## + Fold05.Rep1: alpha=0.18579, lambda=0.002510 
## - Fold05.Rep1: alpha=0.18579, lambda=0.002510 
## + Fold05.Rep1: alpha=0.36092, lambda=0.018533 
## - Fold05.Rep1: alpha=0.36092, lambda=0.018533 
## + Fold06.Rep1: alpha=0.30285, lambda=0.004666 
## - Fold06.Rep1: alpha=0.30285, lambda=0.004666 
## + Fold06.Rep1: alpha=0.02355, lambda=0.061703 
## - Fold06.Rep1: alpha=0.02355, lambda=0.061703 
## + Fold06.Rep1: alpha=0.32050, lambda=0.005575 
## - Fold06.Rep1: alpha=0.32050, lambda=0.005575 
## + Fold06.Rep1: alpha=0.03179, lambda=0.019563 
## - Fold06.Rep1: alpha=0.03179, lambda=0.019563 
## + Fold06.Rep1: alpha=0.55008, lambda=0.600447 
## - Fold06.Rep1: alpha=0.55008, lambda=0.600447 
## + Fold06.Rep1: alpha=0.71987, lambda=0.017336 
## - Fold06.Rep1: alpha=0.71987, lambda=0.017336 
## + Fold06.Rep1: alpha=0.16961, lambda=0.086625 
## - Fold06.Rep1: alpha=0.16961, lambda=0.086625 
## + Fold06.Rep1: alpha=0.61948, lambda=0.011854 
## - Fold06.Rep1: alpha=0.61948, lambda=0.011854 
## + Fold06.Rep1: alpha=0.18579, lambda=0.002510 
## - Fold06.Rep1: alpha=0.18579, lambda=0.002510 
## + Fold06.Rep1: alpha=0.36092, lambda=0.018533 
## - Fold06.Rep1: alpha=0.36092, lambda=0.018533 
## + Fold07.Rep1: alpha=0.30285, lambda=0.004666 
## - Fold07.Rep1: alpha=0.30285, lambda=0.004666 
## + Fold07.Rep1: alpha=0.02355, lambda=0.061703 
## - Fold07.Rep1: alpha=0.02355, lambda=0.061703 
## + Fold07.Rep1: alpha=0.32050, lambda=0.005575 
## - Fold07.Rep1: alpha=0.32050, lambda=0.005575 
## + Fold07.Rep1: alpha=0.03179, lambda=0.019563 
## - Fold07.Rep1: alpha=0.03179, lambda=0.019563 
## + Fold07.Rep1: alpha=0.55008, lambda=0.600447 
## - Fold07.Rep1: alpha=0.55008, lambda=0.600447 
## + Fold07.Rep1: alpha=0.71987, lambda=0.017336 
## - Fold07.Rep1: alpha=0.71987, lambda=0.017336 
## + Fold07.Rep1: alpha=0.16961, lambda=0.086625 
## - Fold07.Rep1: alpha=0.16961, lambda=0.086625 
## + Fold07.Rep1: alpha=0.61948, lambda=0.011854 
## - Fold07.Rep1: alpha=0.61948, lambda=0.011854 
## + Fold07.Rep1: alpha=0.18579, lambda=0.002510 
## - Fold07.Rep1: alpha=0.18579, lambda=0.002510 
## + Fold07.Rep1: alpha=0.36092, lambda=0.018533 
## - Fold07.Rep1: alpha=0.36092, lambda=0.018533 
## + Fold08.Rep1: alpha=0.30285, lambda=0.004666 
## - Fold08.Rep1: alpha=0.30285, lambda=0.004666 
## + Fold08.Rep1: alpha=0.02355, lambda=0.061703 
## - Fold08.Rep1: alpha=0.02355, lambda=0.061703 
## + Fold08.Rep1: alpha=0.32050, lambda=0.005575 
## - Fold08.Rep1: alpha=0.32050, lambda=0.005575 
## + Fold08.Rep1: alpha=0.03179, lambda=0.019563 
## - Fold08.Rep1: alpha=0.03179, lambda=0.019563 
## + Fold08.Rep1: alpha=0.55008, lambda=0.600447 
## - Fold08.Rep1: alpha=0.55008, lambda=0.600447 
## + Fold08.Rep1: alpha=0.71987, lambda=0.017336 
## - Fold08.Rep1: alpha=0.71987, lambda=0.017336 
## + Fold08.Rep1: alpha=0.16961, lambda=0.086625 
## - Fold08.Rep1: alpha=0.16961, lambda=0.086625 
## + Fold08.Rep1: alpha=0.61948, lambda=0.011854 
## - Fold08.Rep1: alpha=0.61948, lambda=0.011854 
## + Fold08.Rep1: alpha=0.18579, lambda=0.002510 
## - Fold08.Rep1: alpha=0.18579, lambda=0.002510 
## + Fold08.Rep1: alpha=0.36092, lambda=0.018533 
## - Fold08.Rep1: alpha=0.36092, lambda=0.018533 
## + Fold09.Rep1: alpha=0.30285, lambda=0.004666 
## - Fold09.Rep1: alpha=0.30285, lambda=0.004666 
## + Fold09.Rep1: alpha=0.02355, lambda=0.061703 
## - Fold09.Rep1: alpha=0.02355, lambda=0.061703 
## + Fold09.Rep1: alpha=0.32050, lambda=0.005575 
## - Fold09.Rep1: alpha=0.32050, lambda=0.005575 
## + Fold09.Rep1: alpha=0.03179, lambda=0.019563 
## - Fold09.Rep1: alpha=0.03179, lambda=0.019563 
## + Fold09.Rep1: alpha=0.55008, lambda=0.600447 
## - Fold09.Rep1: alpha=0.55008, lambda=0.600447 
## + Fold09.Rep1: alpha=0.71987, lambda=0.017336 
## - Fold09.Rep1: alpha=0.71987, lambda=0.017336 
## + Fold09.Rep1: alpha=0.16961, lambda=0.086625 
## - Fold09.Rep1: alpha=0.16961, lambda=0.086625 
## + Fold09.Rep1: alpha=0.61948, lambda=0.011854 
## - Fold09.Rep1: alpha=0.61948, lambda=0.011854 
## + Fold09.Rep1: alpha=0.18579, lambda=0.002510 
## - Fold09.Rep1: alpha=0.18579, lambda=0.002510 
## + Fold09.Rep1: alpha=0.36092, lambda=0.018533 
## - Fold09.Rep1: alpha=0.36092, lambda=0.018533 
## + Fold10.Rep1: alpha=0.30285, lambda=0.004666 
## - Fold10.Rep1: alpha=0.30285, lambda=0.004666 
## + Fold10.Rep1: alpha=0.02355, lambda=0.061703 
## - Fold10.Rep1: alpha=0.02355, lambda=0.061703 
## + Fold10.Rep1: alpha=0.32050, lambda=0.005575 
## - Fold10.Rep1: alpha=0.32050, lambda=0.005575 
## + Fold10.Rep1: alpha=0.03179, lambda=0.019563 
## - Fold10.Rep1: alpha=0.03179, lambda=0.019563 
## + Fold10.Rep1: alpha=0.55008, lambda=0.600447 
## - Fold10.Rep1: alpha=0.55008, lambda=0.600447 
## + Fold10.Rep1: alpha=0.71987, lambda=0.017336 
## - Fold10.Rep1: alpha=0.71987, lambda=0.017336 
## + Fold10.Rep1: alpha=0.16961, lambda=0.086625 
## - Fold10.Rep1: alpha=0.16961, lambda=0.086625 
## + Fold10.Rep1: alpha=0.61948, lambda=0.011854 
## - Fold10.Rep1: alpha=0.61948, lambda=0.011854 
## + Fold10.Rep1: alpha=0.18579, lambda=0.002510 
## - Fold10.Rep1: alpha=0.18579, lambda=0.002510 
## + Fold10.Rep1: alpha=0.36092, lambda=0.018533 
## - Fold10.Rep1: alpha=0.36092, lambda=0.018533 
## + Fold01.Rep2: alpha=0.30285, lambda=0.004666 
## - Fold01.Rep2: alpha=0.30285, lambda=0.004666 
## + Fold01.Rep2: alpha=0.02355, lambda=0.061703 
## - Fold01.Rep2: alpha=0.02355, lambda=0.061703 
## + Fold01.Rep2: alpha=0.32050, lambda=0.005575 
## - Fold01.Rep2: alpha=0.32050, lambda=0.005575 
## + Fold01.Rep2: alpha=0.03179, lambda=0.019563 
## - Fold01.Rep2: alpha=0.03179, lambda=0.019563 
## + Fold01.Rep2: alpha=0.55008, lambda=0.600447 
## - Fold01.Rep2: alpha=0.55008, lambda=0.600447 
## + Fold01.Rep2: alpha=0.71987, lambda=0.017336 
## - Fold01.Rep2: alpha=0.71987, lambda=0.017336 
## + Fold01.Rep2: alpha=0.16961, lambda=0.086625 
## - Fold01.Rep2: alpha=0.16961, lambda=0.086625 
## + Fold01.Rep2: alpha=0.61948, lambda=0.011854 
## - Fold01.Rep2: alpha=0.61948, lambda=0.011854 
## + Fold01.Rep2: alpha=0.18579, lambda=0.002510 
## - Fold01.Rep2: alpha=0.18579, lambda=0.002510 
## + Fold01.Rep2: alpha=0.36092, lambda=0.018533 
## - Fold01.Rep2: alpha=0.36092, lambda=0.018533 
## + Fold02.Rep2: alpha=0.30285, lambda=0.004666 
## - Fold02.Rep2: alpha=0.30285, lambda=0.004666 
## + Fold02.Rep2: alpha=0.02355, lambda=0.061703 
## - Fold02.Rep2: alpha=0.02355, lambda=0.061703 
## + Fold02.Rep2: alpha=0.32050, lambda=0.005575 
## - Fold02.Rep2: alpha=0.32050, lambda=0.005575 
## + Fold02.Rep2: alpha=0.03179, lambda=0.019563 
## - Fold02.Rep2: alpha=0.03179, lambda=0.019563 
## + Fold02.Rep2: alpha=0.55008, lambda=0.600447 
## - Fold02.Rep2: alpha=0.55008, lambda=0.600447 
## + Fold02.Rep2: alpha=0.71987, lambda=0.017336 
## - Fold02.Rep2: alpha=0.71987, lambda=0.017336 
## + Fold02.Rep2: alpha=0.16961, lambda=0.086625 
## - Fold02.Rep2: alpha=0.16961, lambda=0.086625 
## + Fold02.Rep2: alpha=0.61948, lambda=0.011854 
## - Fold02.Rep2: alpha=0.61948, lambda=0.011854 
## + Fold02.Rep2: alpha=0.18579, lambda=0.002510 
## - Fold02.Rep2: alpha=0.18579, lambda=0.002510 
## + Fold02.Rep2: alpha=0.36092, lambda=0.018533 
## - Fold02.Rep2: alpha=0.36092, lambda=0.018533 
## + Fold03.Rep2: alpha=0.30285, lambda=0.004666 
## - Fold03.Rep2: alpha=0.30285, lambda=0.004666 
## + Fold03.Rep2: alpha=0.02355, lambda=0.061703 
## - Fold03.Rep2: alpha=0.02355, lambda=0.061703 
## + Fold03.Rep2: alpha=0.32050, lambda=0.005575 
## - Fold03.Rep2: alpha=0.32050, lambda=0.005575 
## + Fold03.Rep2: alpha=0.03179, lambda=0.019563 
## - Fold03.Rep2: alpha=0.03179, lambda=0.019563 
## + Fold03.Rep2: alpha=0.55008, lambda=0.600447 
## - Fold03.Rep2: alpha=0.55008, lambda=0.600447 
## + Fold03.Rep2: alpha=0.71987, lambda=0.017336 
## - Fold03.Rep2: alpha=0.71987, lambda=0.017336 
## + Fold03.Rep2: alpha=0.16961, lambda=0.086625 
## - Fold03.Rep2: alpha=0.16961, lambda=0.086625 
## + Fold03.Rep2: alpha=0.61948, lambda=0.011854 
## - Fold03.Rep2: alpha=0.61948, lambda=0.011854 
## + Fold03.Rep2: alpha=0.18579, lambda=0.002510 
## - Fold03.Rep2: alpha=0.18579, lambda=0.002510 
## + Fold03.Rep2: alpha=0.36092, lambda=0.018533 
## - Fold03.Rep2: alpha=0.36092, lambda=0.018533 
## + Fold04.Rep2: alpha=0.30285, lambda=0.004666 
## - Fold04.Rep2: alpha=0.30285, lambda=0.004666 
## + Fold04.Rep2: alpha=0.02355, lambda=0.061703 
## - Fold04.Rep2: alpha=0.02355, lambda=0.061703 
## + Fold04.Rep2: alpha=0.32050, lambda=0.005575 
## - Fold04.Rep2: alpha=0.32050, lambda=0.005575 
## + Fold04.Rep2: alpha=0.03179, lambda=0.019563 
## - Fold04.Rep2: alpha=0.03179, lambda=0.019563 
## + Fold04.Rep2: alpha=0.55008, lambda=0.600447 
## - Fold04.Rep2: alpha=0.55008, lambda=0.600447 
## + Fold04.Rep2: alpha=0.71987, lambda=0.017336 
## - Fold04.Rep2: alpha=0.71987, lambda=0.017336 
## + Fold04.Rep2: alpha=0.16961, lambda=0.086625 
## - Fold04.Rep2: alpha=0.16961, lambda=0.086625 
## + Fold04.Rep2: alpha=0.61948, lambda=0.011854 
## - Fold04.Rep2: alpha=0.61948, lambda=0.011854 
## + Fold04.Rep2: alpha=0.18579, lambda=0.002510 
## - Fold04.Rep2: alpha=0.18579, lambda=0.002510 
## + Fold04.Rep2: alpha=0.36092, lambda=0.018533 
## - Fold04.Rep2: alpha=0.36092, lambda=0.018533 
## + Fold05.Rep2: alpha=0.30285, lambda=0.004666 
## - Fold05.Rep2: alpha=0.30285, lambda=0.004666 
## + Fold05.Rep2: alpha=0.02355, lambda=0.061703 
## - Fold05.Rep2: alpha=0.02355, lambda=0.061703 
## + Fold05.Rep2: alpha=0.32050, lambda=0.005575 
## - Fold05.Rep2: alpha=0.32050, lambda=0.005575 
## + Fold05.Rep2: alpha=0.03179, lambda=0.019563 
## - Fold05.Rep2: alpha=0.03179, lambda=0.019563 
## + Fold05.Rep2: alpha=0.55008, lambda=0.600447 
## - Fold05.Rep2: alpha=0.55008, lambda=0.600447 
## + Fold05.Rep2: alpha=0.71987, lambda=0.017336 
## - Fold05.Rep2: alpha=0.71987, lambda=0.017336 
## + Fold05.Rep2: alpha=0.16961, lambda=0.086625 
## - Fold05.Rep2: alpha=0.16961, lambda=0.086625 
## + Fold05.Rep2: alpha=0.61948, lambda=0.011854 
## - Fold05.Rep2: alpha=0.61948, lambda=0.011854 
## + Fold05.Rep2: alpha=0.18579, lambda=0.002510 
## - Fold05.Rep2: alpha=0.18579, lambda=0.002510 
## + Fold05.Rep2: alpha=0.36092, lambda=0.018533 
## - Fold05.Rep2: alpha=0.36092, lambda=0.018533 
## + Fold06.Rep2: alpha=0.30285, lambda=0.004666 
## - Fold06.Rep2: alpha=0.30285, lambda=0.004666 
## + Fold06.Rep2: alpha=0.02355, lambda=0.061703 
## - Fold06.Rep2: alpha=0.02355, lambda=0.061703 
## + Fold06.Rep2: alpha=0.32050, lambda=0.005575 
## - Fold06.Rep2: alpha=0.32050, lambda=0.005575 
## + Fold06.Rep2: alpha=0.03179, lambda=0.019563 
## - Fold06.Rep2: alpha=0.03179, lambda=0.019563 
## + Fold06.Rep2: alpha=0.55008, lambda=0.600447 
## - Fold06.Rep2: alpha=0.55008, lambda=0.600447 
## + Fold06.Rep2: alpha=0.71987, lambda=0.017336 
## - Fold06.Rep2: alpha=0.71987, lambda=0.017336 
## + Fold06.Rep2: alpha=0.16961, lambda=0.086625 
## - Fold06.Rep2: alpha=0.16961, lambda=0.086625 
## + Fold06.Rep2: alpha=0.61948, lambda=0.011854 
## - Fold06.Rep2: alpha=0.61948, lambda=0.011854 
## + Fold06.Rep2: alpha=0.18579, lambda=0.002510 
## - Fold06.Rep2: alpha=0.18579, lambda=0.002510 
## + Fold06.Rep2: alpha=0.36092, lambda=0.018533 
## - Fold06.Rep2: alpha=0.36092, lambda=0.018533 
## + Fold07.Rep2: alpha=0.30285, lambda=0.004666 
## - Fold07.Rep2: alpha=0.30285, lambda=0.004666 
## + Fold07.Rep2: alpha=0.02355, lambda=0.061703 
## - Fold07.Rep2: alpha=0.02355, lambda=0.061703 
## + Fold07.Rep2: alpha=0.32050, lambda=0.005575 
## - Fold07.Rep2: alpha=0.32050, lambda=0.005575 
## + Fold07.Rep2: alpha=0.03179, lambda=0.019563 
## - Fold07.Rep2: alpha=0.03179, lambda=0.019563 
## + Fold07.Rep2: alpha=0.55008, lambda=0.600447 
## - Fold07.Rep2: alpha=0.55008, lambda=0.600447 
## + Fold07.Rep2: alpha=0.71987, lambda=0.017336 
## - Fold07.Rep2: alpha=0.71987, lambda=0.017336 
## + Fold07.Rep2: alpha=0.16961, lambda=0.086625 
## - Fold07.Rep2: alpha=0.16961, lambda=0.086625 
## + Fold07.Rep2: alpha=0.61948, lambda=0.011854 
## - Fold07.Rep2: alpha=0.61948, lambda=0.011854 
## + Fold07.Rep2: alpha=0.18579, lambda=0.002510 
## - Fold07.Rep2: alpha=0.18579, lambda=0.002510 
## + Fold07.Rep2: alpha=0.36092, lambda=0.018533 
## - Fold07.Rep2: alpha=0.36092, lambda=0.018533 
## + Fold08.Rep2: alpha=0.30285, lambda=0.004666 
## - Fold08.Rep2: alpha=0.30285, lambda=0.004666 
## + Fold08.Rep2: alpha=0.02355, lambda=0.061703 
## - Fold08.Rep2: alpha=0.02355, lambda=0.061703 
## + Fold08.Rep2: alpha=0.32050, lambda=0.005575 
## - Fold08.Rep2: alpha=0.32050, lambda=0.005575 
## + Fold08.Rep2: alpha=0.03179, lambda=0.019563 
## - Fold08.Rep2: alpha=0.03179, lambda=0.019563 
## + Fold08.Rep2: alpha=0.55008, lambda=0.600447 
## - Fold08.Rep2: alpha=0.55008, lambda=0.600447 
## + Fold08.Rep2: alpha=0.71987, lambda=0.017336 
## - Fold08.Rep2: alpha=0.71987, lambda=0.017336 
## + Fold08.Rep2: alpha=0.16961, lambda=0.086625 
## - Fold08.Rep2: alpha=0.16961, lambda=0.086625 
## + Fold08.Rep2: alpha=0.61948, lambda=0.011854 
## - Fold08.Rep2: alpha=0.61948, lambda=0.011854 
## + Fold08.Rep2: alpha=0.18579, lambda=0.002510 
## - Fold08.Rep2: alpha=0.18579, lambda=0.002510 
## + Fold08.Rep2: alpha=0.36092, lambda=0.018533 
## - Fold08.Rep2: alpha=0.36092, lambda=0.018533 
## + Fold09.Rep2: alpha=0.30285, lambda=0.004666 
## - Fold09.Rep2: alpha=0.30285, lambda=0.004666 
## + Fold09.Rep2: alpha=0.02355, lambda=0.061703 
## - Fold09.Rep2: alpha=0.02355, lambda=0.061703 
## + Fold09.Rep2: alpha=0.32050, lambda=0.005575 
## - Fold09.Rep2: alpha=0.32050, lambda=0.005575 
## + Fold09.Rep2: alpha=0.03179, lambda=0.019563 
## - Fold09.Rep2: alpha=0.03179, lambda=0.019563 
## + Fold09.Rep2: alpha=0.55008, lambda=0.600447 
## - Fold09.Rep2: alpha=0.55008, lambda=0.600447 
## + Fold09.Rep2: alpha=0.71987, lambda=0.017336 
## - Fold09.Rep2: alpha=0.71987, lambda=0.017336 
## + Fold09.Rep2: alpha=0.16961, lambda=0.086625 
## - Fold09.Rep2: alpha=0.16961, lambda=0.086625 
## + Fold09.Rep2: alpha=0.61948, lambda=0.011854 
## - Fold09.Rep2: alpha=0.61948, lambda=0.011854 
## + Fold09.Rep2: alpha=0.18579, lambda=0.002510 
## - Fold09.Rep2: alpha=0.18579, lambda=0.002510 
## + Fold09.Rep2: alpha=0.36092, lambda=0.018533 
## - Fold09.Rep2: alpha=0.36092, lambda=0.018533 
## + Fold10.Rep2: alpha=0.30285, lambda=0.004666 
## - Fold10.Rep2: alpha=0.30285, lambda=0.004666 
## + Fold10.Rep2: alpha=0.02355, lambda=0.061703 
## - Fold10.Rep2: alpha=0.02355, lambda=0.061703 
## + Fold10.Rep2: alpha=0.32050, lambda=0.005575 
## - Fold10.Rep2: alpha=0.32050, lambda=0.005575 
## + Fold10.Rep2: alpha=0.03179, lambda=0.019563 
## - Fold10.Rep2: alpha=0.03179, lambda=0.019563 
## + Fold10.Rep2: alpha=0.55008, lambda=0.600447 
## - Fold10.Rep2: alpha=0.55008, lambda=0.600447 
## + Fold10.Rep2: alpha=0.71987, lambda=0.017336 
## - Fold10.Rep2: alpha=0.71987, lambda=0.017336 
## + Fold10.Rep2: alpha=0.16961, lambda=0.086625 
## - Fold10.Rep2: alpha=0.16961, lambda=0.086625 
## + Fold10.Rep2: alpha=0.61948, lambda=0.011854 
## - Fold10.Rep2: alpha=0.61948, lambda=0.011854 
## + Fold10.Rep2: alpha=0.18579, lambda=0.002510 
## - Fold10.Rep2: alpha=0.18579, lambda=0.002510 
## + Fold10.Rep2: alpha=0.36092, lambda=0.018533 
## - Fold10.Rep2: alpha=0.36092, lambda=0.018533 
## + Fold01.Rep3: alpha=0.30285, lambda=0.004666 
## - Fold01.Rep3: alpha=0.30285, lambda=0.004666 
## + Fold01.Rep3: alpha=0.02355, lambda=0.061703 
## - Fold01.Rep3: alpha=0.02355, lambda=0.061703 
## + Fold01.Rep3: alpha=0.32050, lambda=0.005575 
## - Fold01.Rep3: alpha=0.32050, lambda=0.005575 
## + Fold01.Rep3: alpha=0.03179, lambda=0.019563 
## - Fold01.Rep3: alpha=0.03179, lambda=0.019563 
## + Fold01.Rep3: alpha=0.55008, lambda=0.600447 
## - Fold01.Rep3: alpha=0.55008, lambda=0.600447 
## + Fold01.Rep3: alpha=0.71987, lambda=0.017336 
## - Fold01.Rep3: alpha=0.71987, lambda=0.017336 
## + Fold01.Rep3: alpha=0.16961, lambda=0.086625 
## - Fold01.Rep3: alpha=0.16961, lambda=0.086625 
## + Fold01.Rep3: alpha=0.61948, lambda=0.011854 
## - Fold01.Rep3: alpha=0.61948, lambda=0.011854 
## + Fold01.Rep3: alpha=0.18579, lambda=0.002510 
## - Fold01.Rep3: alpha=0.18579, lambda=0.002510 
## + Fold01.Rep3: alpha=0.36092, lambda=0.018533 
## - Fold01.Rep3: alpha=0.36092, lambda=0.018533 
## + Fold02.Rep3: alpha=0.30285, lambda=0.004666 
## - Fold02.Rep3: alpha=0.30285, lambda=0.004666 
## + Fold02.Rep3: alpha=0.02355, lambda=0.061703 
## - Fold02.Rep3: alpha=0.02355, lambda=0.061703 
## + Fold02.Rep3: alpha=0.32050, lambda=0.005575 
## - Fold02.Rep3: alpha=0.32050, lambda=0.005575 
## + Fold02.Rep3: alpha=0.03179, lambda=0.019563 
## - Fold02.Rep3: alpha=0.03179, lambda=0.019563 
## + Fold02.Rep3: alpha=0.55008, lambda=0.600447 
## - Fold02.Rep3: alpha=0.55008, lambda=0.600447 
## + Fold02.Rep3: alpha=0.71987, lambda=0.017336 
## - Fold02.Rep3: alpha=0.71987, lambda=0.017336 
## + Fold02.Rep3: alpha=0.16961, lambda=0.086625 
## - Fold02.Rep3: alpha=0.16961, lambda=0.086625 
## + Fold02.Rep3: alpha=0.61948, lambda=0.011854 
## - Fold02.Rep3: alpha=0.61948, lambda=0.011854 
## + Fold02.Rep3: alpha=0.18579, lambda=0.002510 
## - Fold02.Rep3: alpha=0.18579, lambda=0.002510 
## + Fold02.Rep3: alpha=0.36092, lambda=0.018533 
## - Fold02.Rep3: alpha=0.36092, lambda=0.018533 
## + Fold03.Rep3: alpha=0.30285, lambda=0.004666 
## - Fold03.Rep3: alpha=0.30285, lambda=0.004666 
## + Fold03.Rep3: alpha=0.02355, lambda=0.061703 
## - Fold03.Rep3: alpha=0.02355, lambda=0.061703 
## + Fold03.Rep3: alpha=0.32050, lambda=0.005575 
## - Fold03.Rep3: alpha=0.32050, lambda=0.005575 
## + Fold03.Rep3: alpha=0.03179, lambda=0.019563 
## - Fold03.Rep3: alpha=0.03179, lambda=0.019563 
## + Fold03.Rep3: alpha=0.55008, lambda=0.600447 
## - Fold03.Rep3: alpha=0.55008, lambda=0.600447 
## + Fold03.Rep3: alpha=0.71987, lambda=0.017336 
## - Fold03.Rep3: alpha=0.71987, lambda=0.017336 
## + Fold03.Rep3: alpha=0.16961, lambda=0.086625 
## - Fold03.Rep3: alpha=0.16961, lambda=0.086625 
## + Fold03.Rep3: alpha=0.61948, lambda=0.011854 
## - Fold03.Rep3: alpha=0.61948, lambda=0.011854 
## + Fold03.Rep3: alpha=0.18579, lambda=0.002510 
## - Fold03.Rep3: alpha=0.18579, lambda=0.002510 
## + Fold03.Rep3: alpha=0.36092, lambda=0.018533 
## - Fold03.Rep3: alpha=0.36092, lambda=0.018533 
## + Fold04.Rep3: alpha=0.30285, lambda=0.004666 
## - Fold04.Rep3: alpha=0.30285, lambda=0.004666 
## + Fold04.Rep3: alpha=0.02355, lambda=0.061703 
## - Fold04.Rep3: alpha=0.02355, lambda=0.061703 
## + Fold04.Rep3: alpha=0.32050, lambda=0.005575 
## - Fold04.Rep3: alpha=0.32050, lambda=0.005575 
## + Fold04.Rep3: alpha=0.03179, lambda=0.019563 
## - Fold04.Rep3: alpha=0.03179, lambda=0.019563 
## + Fold04.Rep3: alpha=0.55008, lambda=0.600447 
## - Fold04.Rep3: alpha=0.55008, lambda=0.600447 
## + Fold04.Rep3: alpha=0.71987, lambda=0.017336 
## - Fold04.Rep3: alpha=0.71987, lambda=0.017336 
## + Fold04.Rep3: alpha=0.16961, lambda=0.086625 
## - Fold04.Rep3: alpha=0.16961, lambda=0.086625 
## + Fold04.Rep3: alpha=0.61948, lambda=0.011854 
## - Fold04.Rep3: alpha=0.61948, lambda=0.011854 
## + Fold04.Rep3: alpha=0.18579, lambda=0.002510 
## - Fold04.Rep3: alpha=0.18579, lambda=0.002510 
## + Fold04.Rep3: alpha=0.36092, lambda=0.018533 
## - Fold04.Rep3: alpha=0.36092, lambda=0.018533 
## + Fold05.Rep3: alpha=0.30285, lambda=0.004666 
## - Fold05.Rep3: alpha=0.30285, lambda=0.004666 
## + Fold05.Rep3: alpha=0.02355, lambda=0.061703 
## - Fold05.Rep3: alpha=0.02355, lambda=0.061703 
## + Fold05.Rep3: alpha=0.32050, lambda=0.005575 
## - Fold05.Rep3: alpha=0.32050, lambda=0.005575 
## + Fold05.Rep3: alpha=0.03179, lambda=0.019563 
## - Fold05.Rep3: alpha=0.03179, lambda=0.019563 
## + Fold05.Rep3: alpha=0.55008, lambda=0.600447 
## - Fold05.Rep3: alpha=0.55008, lambda=0.600447 
## + Fold05.Rep3: alpha=0.71987, lambda=0.017336 
## - Fold05.Rep3: alpha=0.71987, lambda=0.017336 
## + Fold05.Rep3: alpha=0.16961, lambda=0.086625 
## - Fold05.Rep3: alpha=0.16961, lambda=0.086625 
## + Fold05.Rep3: alpha=0.61948, lambda=0.011854 
## - Fold05.Rep3: alpha=0.61948, lambda=0.011854 
## + Fold05.Rep3: alpha=0.18579, lambda=0.002510 
## - Fold05.Rep3: alpha=0.18579, lambda=0.002510 
## + Fold05.Rep3: alpha=0.36092, lambda=0.018533 
## - Fold05.Rep3: alpha=0.36092, lambda=0.018533 
## + Fold06.Rep3: alpha=0.30285, lambda=0.004666 
## - Fold06.Rep3: alpha=0.30285, lambda=0.004666 
## + Fold06.Rep3: alpha=0.02355, lambda=0.061703 
## - Fold06.Rep3: alpha=0.02355, lambda=0.061703 
## + Fold06.Rep3: alpha=0.32050, lambda=0.005575 
## - Fold06.Rep3: alpha=0.32050, lambda=0.005575 
## + Fold06.Rep3: alpha=0.03179, lambda=0.019563 
## - Fold06.Rep3: alpha=0.03179, lambda=0.019563 
## + Fold06.Rep3: alpha=0.55008, lambda=0.600447 
## - Fold06.Rep3: alpha=0.55008, lambda=0.600447 
## + Fold06.Rep3: alpha=0.71987, lambda=0.017336 
## - Fold06.Rep3: alpha=0.71987, lambda=0.017336 
## + Fold06.Rep3: alpha=0.16961, lambda=0.086625 
## - Fold06.Rep3: alpha=0.16961, lambda=0.086625 
## + Fold06.Rep3: alpha=0.61948, lambda=0.011854 
## - Fold06.Rep3: alpha=0.61948, lambda=0.011854 
## + Fold06.Rep3: alpha=0.18579, lambda=0.002510 
## - Fold06.Rep3: alpha=0.18579, lambda=0.002510 
## + Fold06.Rep3: alpha=0.36092, lambda=0.018533 
## - Fold06.Rep3: alpha=0.36092, lambda=0.018533 
## + Fold07.Rep3: alpha=0.30285, lambda=0.004666 
## - Fold07.Rep3: alpha=0.30285, lambda=0.004666 
## + Fold07.Rep3: alpha=0.02355, lambda=0.061703 
## - Fold07.Rep3: alpha=0.02355, lambda=0.061703 
## + Fold07.Rep3: alpha=0.32050, lambda=0.005575 
## - Fold07.Rep3: alpha=0.32050, lambda=0.005575 
## + Fold07.Rep3: alpha=0.03179, lambda=0.019563 
## - Fold07.Rep3: alpha=0.03179, lambda=0.019563 
## + Fold07.Rep3: alpha=0.55008, lambda=0.600447 
## - Fold07.Rep3: alpha=0.55008, lambda=0.600447 
## + Fold07.Rep3: alpha=0.71987, lambda=0.017336 
## - Fold07.Rep3: alpha=0.71987, lambda=0.017336 
## + Fold07.Rep3: alpha=0.16961, lambda=0.086625 
## - Fold07.Rep3: alpha=0.16961, lambda=0.086625 
## + Fold07.Rep3: alpha=0.61948, lambda=0.011854 
## - Fold07.Rep3: alpha=0.61948, lambda=0.011854 
## + Fold07.Rep3: alpha=0.18579, lambda=0.002510 
## - Fold07.Rep3: alpha=0.18579, lambda=0.002510 
## + Fold07.Rep3: alpha=0.36092, lambda=0.018533 
## - Fold07.Rep3: alpha=0.36092, lambda=0.018533 
## + Fold08.Rep3: alpha=0.30285, lambda=0.004666 
## - Fold08.Rep3: alpha=0.30285, lambda=0.004666 
## + Fold08.Rep3: alpha=0.02355, lambda=0.061703 
## - Fold08.Rep3: alpha=0.02355, lambda=0.061703 
## + Fold08.Rep3: alpha=0.32050, lambda=0.005575 
## - Fold08.Rep3: alpha=0.32050, lambda=0.005575 
## + Fold08.Rep3: alpha=0.03179, lambda=0.019563 
## - Fold08.Rep3: alpha=0.03179, lambda=0.019563 
## + Fold08.Rep3: alpha=0.55008, lambda=0.600447 
## - Fold08.Rep3: alpha=0.55008, lambda=0.600447 
## + Fold08.Rep3: alpha=0.71987, lambda=0.017336 
## - Fold08.Rep3: alpha=0.71987, lambda=0.017336 
## + Fold08.Rep3: alpha=0.16961, lambda=0.086625 
## - Fold08.Rep3: alpha=0.16961, lambda=0.086625 
## + Fold08.Rep3: alpha=0.61948, lambda=0.011854 
## - Fold08.Rep3: alpha=0.61948, lambda=0.011854 
## + Fold08.Rep3: alpha=0.18579, lambda=0.002510 
## - Fold08.Rep3: alpha=0.18579, lambda=0.002510 
## + Fold08.Rep3: alpha=0.36092, lambda=0.018533 
## - Fold08.Rep3: alpha=0.36092, lambda=0.018533 
## + Fold09.Rep3: alpha=0.30285, lambda=0.004666 
## - Fold09.Rep3: alpha=0.30285, lambda=0.004666 
## + Fold09.Rep3: alpha=0.02355, lambda=0.061703 
## - Fold09.Rep3: alpha=0.02355, lambda=0.061703 
## + Fold09.Rep3: alpha=0.32050, lambda=0.005575 
## - Fold09.Rep3: alpha=0.32050, lambda=0.005575 
## + Fold09.Rep3: alpha=0.03179, lambda=0.019563 
## - Fold09.Rep3: alpha=0.03179, lambda=0.019563 
## + Fold09.Rep3: alpha=0.55008, lambda=0.600447 
## - Fold09.Rep3: alpha=0.55008, lambda=0.600447 
## + Fold09.Rep3: alpha=0.71987, lambda=0.017336 
## - Fold09.Rep3: alpha=0.71987, lambda=0.017336 
## + Fold09.Rep3: alpha=0.16961, lambda=0.086625 
## - Fold09.Rep3: alpha=0.16961, lambda=0.086625 
## + Fold09.Rep3: alpha=0.61948, lambda=0.011854 
## - Fold09.Rep3: alpha=0.61948, lambda=0.011854 
## + Fold09.Rep3: alpha=0.18579, lambda=0.002510 
## - Fold09.Rep3: alpha=0.18579, lambda=0.002510 
## + Fold09.Rep3: alpha=0.36092, lambda=0.018533 
## - Fold09.Rep3: alpha=0.36092, lambda=0.018533 
## + Fold10.Rep3: alpha=0.30285, lambda=0.004666 
## - Fold10.Rep3: alpha=0.30285, lambda=0.004666 
## + Fold10.Rep3: alpha=0.02355, lambda=0.061703 
## - Fold10.Rep3: alpha=0.02355, lambda=0.061703 
## + Fold10.Rep3: alpha=0.32050, lambda=0.005575 
## - Fold10.Rep3: alpha=0.32050, lambda=0.005575 
## + Fold10.Rep3: alpha=0.03179, lambda=0.019563 
## - Fold10.Rep3: alpha=0.03179, lambda=0.019563 
## + Fold10.Rep3: alpha=0.55008, lambda=0.600447 
## - Fold10.Rep3: alpha=0.55008, lambda=0.600447 
## + Fold10.Rep3: alpha=0.71987, lambda=0.017336 
## - Fold10.Rep3: alpha=0.71987, lambda=0.017336 
## + Fold10.Rep3: alpha=0.16961, lambda=0.086625 
## - Fold10.Rep3: alpha=0.16961, lambda=0.086625 
## + Fold10.Rep3: alpha=0.61948, lambda=0.011854 
## - Fold10.Rep3: alpha=0.61948, lambda=0.011854 
## + Fold10.Rep3: alpha=0.18579, lambda=0.002510 
## - Fold10.Rep3: alpha=0.18579, lambda=0.002510 
## + Fold10.Rep3: alpha=0.36092, lambda=0.018533 
## - Fold10.Rep3: alpha=0.36092, lambda=0.018533 
## + Fold01.Rep4: alpha=0.30285, lambda=0.004666 
## - Fold01.Rep4: alpha=0.30285, lambda=0.004666 
## + Fold01.Rep4: alpha=0.02355, lambda=0.061703 
## - Fold01.Rep4: alpha=0.02355, lambda=0.061703 
## + Fold01.Rep4: alpha=0.32050, lambda=0.005575 
## - Fold01.Rep4: alpha=0.32050, lambda=0.005575 
## + Fold01.Rep4: alpha=0.03179, lambda=0.019563 
## - Fold01.Rep4: alpha=0.03179, lambda=0.019563 
## + Fold01.Rep4: alpha=0.55008, lambda=0.600447 
## - Fold01.Rep4: alpha=0.55008, lambda=0.600447 
## + Fold01.Rep4: alpha=0.71987, lambda=0.017336 
## - Fold01.Rep4: alpha=0.71987, lambda=0.017336 
## + Fold01.Rep4: alpha=0.16961, lambda=0.086625 
## - Fold01.Rep4: alpha=0.16961, lambda=0.086625 
## + Fold01.Rep4: alpha=0.61948, lambda=0.011854 
## - Fold01.Rep4: alpha=0.61948, lambda=0.011854 
## + Fold01.Rep4: alpha=0.18579, lambda=0.002510 
## - Fold01.Rep4: alpha=0.18579, lambda=0.002510 
## + Fold01.Rep4: alpha=0.36092, lambda=0.018533 
## - Fold01.Rep4: alpha=0.36092, lambda=0.018533 
## + Fold02.Rep4: alpha=0.30285, lambda=0.004666 
## - Fold02.Rep4: alpha=0.30285, lambda=0.004666 
## + Fold02.Rep4: alpha=0.02355, lambda=0.061703 
## - Fold02.Rep4: alpha=0.02355, lambda=0.061703 
## + Fold02.Rep4: alpha=0.32050, lambda=0.005575 
## - Fold02.Rep4: alpha=0.32050, lambda=0.005575 
## + Fold02.Rep4: alpha=0.03179, lambda=0.019563 
## - Fold02.Rep4: alpha=0.03179, lambda=0.019563 
## + Fold02.Rep4: alpha=0.55008, lambda=0.600447 
## - Fold02.Rep4: alpha=0.55008, lambda=0.600447 
## + Fold02.Rep4: alpha=0.71987, lambda=0.017336 
## - Fold02.Rep4: alpha=0.71987, lambda=0.017336 
## + Fold02.Rep4: alpha=0.16961, lambda=0.086625 
## - Fold02.Rep4: alpha=0.16961, lambda=0.086625 
## + Fold02.Rep4: alpha=0.61948, lambda=0.011854 
## - Fold02.Rep4: alpha=0.61948, lambda=0.011854 
## + Fold02.Rep4: alpha=0.18579, lambda=0.002510 
## - Fold02.Rep4: alpha=0.18579, lambda=0.002510 
## + Fold02.Rep4: alpha=0.36092, lambda=0.018533 
## - Fold02.Rep4: alpha=0.36092, lambda=0.018533 
## + Fold03.Rep4: alpha=0.30285, lambda=0.004666 
## - Fold03.Rep4: alpha=0.30285, lambda=0.004666 
## + Fold03.Rep4: alpha=0.02355, lambda=0.061703 
## - Fold03.Rep4: alpha=0.02355, lambda=0.061703 
## + Fold03.Rep4: alpha=0.32050, lambda=0.005575 
## - Fold03.Rep4: alpha=0.32050, lambda=0.005575 
## + Fold03.Rep4: alpha=0.03179, lambda=0.019563 
## - Fold03.Rep4: alpha=0.03179, lambda=0.019563 
## + Fold03.Rep4: alpha=0.55008, lambda=0.600447 
## - Fold03.Rep4: alpha=0.55008, lambda=0.600447 
## + Fold03.Rep4: alpha=0.71987, lambda=0.017336 
## - Fold03.Rep4: alpha=0.71987, lambda=0.017336 
## + Fold03.Rep4: alpha=0.16961, lambda=0.086625 
## - Fold03.Rep4: alpha=0.16961, lambda=0.086625 
## + Fold03.Rep4: alpha=0.61948, lambda=0.011854 
## - Fold03.Rep4: alpha=0.61948, lambda=0.011854 
## + Fold03.Rep4: alpha=0.18579, lambda=0.002510 
## - Fold03.Rep4: alpha=0.18579, lambda=0.002510 
## + Fold03.Rep4: alpha=0.36092, lambda=0.018533 
## - Fold03.Rep4: alpha=0.36092, lambda=0.018533 
## + Fold04.Rep4: alpha=0.30285, lambda=0.004666 
## - Fold04.Rep4: alpha=0.30285, lambda=0.004666 
## + Fold04.Rep4: alpha=0.02355, lambda=0.061703 
## - Fold04.Rep4: alpha=0.02355, lambda=0.061703 
## + Fold04.Rep4: alpha=0.32050, lambda=0.005575 
## - Fold04.Rep4: alpha=0.32050, lambda=0.005575 
## + Fold04.Rep4: alpha=0.03179, lambda=0.019563 
## - Fold04.Rep4: alpha=0.03179, lambda=0.019563 
## + Fold04.Rep4: alpha=0.55008, lambda=0.600447 
## - Fold04.Rep4: alpha=0.55008, lambda=0.600447 
## + Fold04.Rep4: alpha=0.71987, lambda=0.017336 
## - Fold04.Rep4: alpha=0.71987, lambda=0.017336 
## + Fold04.Rep4: alpha=0.16961, lambda=0.086625 
## - Fold04.Rep4: alpha=0.16961, lambda=0.086625 
## + Fold04.Rep4: alpha=0.61948, lambda=0.011854 
## - Fold04.Rep4: alpha=0.61948, lambda=0.011854 
## + Fold04.Rep4: alpha=0.18579, lambda=0.002510 
## - Fold04.Rep4: alpha=0.18579, lambda=0.002510 
## + Fold04.Rep4: alpha=0.36092, lambda=0.018533 
## - Fold04.Rep4: alpha=0.36092, lambda=0.018533 
## + Fold05.Rep4: alpha=0.30285, lambda=0.004666 
## - Fold05.Rep4: alpha=0.30285, lambda=0.004666 
## + Fold05.Rep4: alpha=0.02355, lambda=0.061703 
## - Fold05.Rep4: alpha=0.02355, lambda=0.061703 
## + Fold05.Rep4: alpha=0.32050, lambda=0.005575 
## - Fold05.Rep4: alpha=0.32050, lambda=0.005575 
## + Fold05.Rep4: alpha=0.03179, lambda=0.019563 
## - Fold05.Rep4: alpha=0.03179, lambda=0.019563 
## + Fold05.Rep4: alpha=0.55008, lambda=0.600447 
## - Fold05.Rep4: alpha=0.55008, lambda=0.600447 
## + Fold05.Rep4: alpha=0.71987, lambda=0.017336 
## - Fold05.Rep4: alpha=0.71987, lambda=0.017336 
## + Fold05.Rep4: alpha=0.16961, lambda=0.086625 
## - Fold05.Rep4: alpha=0.16961, lambda=0.086625 
## + Fold05.Rep4: alpha=0.61948, lambda=0.011854 
## - Fold05.Rep4: alpha=0.61948, lambda=0.011854 
## + Fold05.Rep4: alpha=0.18579, lambda=0.002510 
## - Fold05.Rep4: alpha=0.18579, lambda=0.002510 
## + Fold05.Rep4: alpha=0.36092, lambda=0.018533 
## - Fold05.Rep4: alpha=0.36092, lambda=0.018533 
## + Fold06.Rep4: alpha=0.30285, lambda=0.004666 
## - Fold06.Rep4: alpha=0.30285, lambda=0.004666 
## + Fold06.Rep4: alpha=0.02355, lambda=0.061703 
## - Fold06.Rep4: alpha=0.02355, lambda=0.061703 
## + Fold06.Rep4: alpha=0.32050, lambda=0.005575 
## - Fold06.Rep4: alpha=0.32050, lambda=0.005575 
## + Fold06.Rep4: alpha=0.03179, lambda=0.019563 
## - Fold06.Rep4: alpha=0.03179, lambda=0.019563 
## + Fold06.Rep4: alpha=0.55008, lambda=0.600447 
## - Fold06.Rep4: alpha=0.55008, lambda=0.600447 
## + Fold06.Rep4: alpha=0.71987, lambda=0.017336 
## - Fold06.Rep4: alpha=0.71987, lambda=0.017336 
## + Fold06.Rep4: alpha=0.16961, lambda=0.086625 
## - Fold06.Rep4: alpha=0.16961, lambda=0.086625 
## + Fold06.Rep4: alpha=0.61948, lambda=0.011854 
## - Fold06.Rep4: alpha=0.61948, lambda=0.011854 
## + Fold06.Rep4: alpha=0.18579, lambda=0.002510 
## - Fold06.Rep4: alpha=0.18579, lambda=0.002510 
## + Fold06.Rep4: alpha=0.36092, lambda=0.018533 
## - Fold06.Rep4: alpha=0.36092, lambda=0.018533 
## + Fold07.Rep4: alpha=0.30285, lambda=0.004666 
## - Fold07.Rep4: alpha=0.30285, lambda=0.004666 
## + Fold07.Rep4: alpha=0.02355, lambda=0.061703 
## - Fold07.Rep4: alpha=0.02355, lambda=0.061703 
## + Fold07.Rep4: alpha=0.32050, lambda=0.005575 
## - Fold07.Rep4: alpha=0.32050, lambda=0.005575 
## + Fold07.Rep4: alpha=0.03179, lambda=0.019563 
## - Fold07.Rep4: alpha=0.03179, lambda=0.019563 
## + Fold07.Rep4: alpha=0.55008, lambda=0.600447 
## - Fold07.Rep4: alpha=0.55008, lambda=0.600447 
## + Fold07.Rep4: alpha=0.71987, lambda=0.017336 
## - Fold07.Rep4: alpha=0.71987, lambda=0.017336 
## + Fold07.Rep4: alpha=0.16961, lambda=0.086625 
## - Fold07.Rep4: alpha=0.16961, lambda=0.086625 
## + Fold07.Rep4: alpha=0.61948, lambda=0.011854 
## - Fold07.Rep4: alpha=0.61948, lambda=0.011854 
## + Fold07.Rep4: alpha=0.18579, lambda=0.002510 
## - Fold07.Rep4: alpha=0.18579, lambda=0.002510 
## + Fold07.Rep4: alpha=0.36092, lambda=0.018533 
## - Fold07.Rep4: alpha=0.36092, lambda=0.018533 
## + Fold08.Rep4: alpha=0.30285, lambda=0.004666 
## - Fold08.Rep4: alpha=0.30285, lambda=0.004666 
## + Fold08.Rep4: alpha=0.02355, lambda=0.061703 
## - Fold08.Rep4: alpha=0.02355, lambda=0.061703 
## + Fold08.Rep4: alpha=0.32050, lambda=0.005575 
## - Fold08.Rep4: alpha=0.32050, lambda=0.005575 
## + Fold08.Rep4: alpha=0.03179, lambda=0.019563 
## - Fold08.Rep4: alpha=0.03179, lambda=0.019563 
## + Fold08.Rep4: alpha=0.55008, lambda=0.600447 
## - Fold08.Rep4: alpha=0.55008, lambda=0.600447 
## + Fold08.Rep4: alpha=0.71987, lambda=0.017336 
## - Fold08.Rep4: alpha=0.71987, lambda=0.017336 
## + Fold08.Rep4: alpha=0.16961, lambda=0.086625 
## - Fold08.Rep4: alpha=0.16961, lambda=0.086625 
## + Fold08.Rep4: alpha=0.61948, lambda=0.011854 
## - Fold08.Rep4: alpha=0.61948, lambda=0.011854 
## + Fold08.Rep4: alpha=0.18579, lambda=0.002510 
## - Fold08.Rep4: alpha=0.18579, lambda=0.002510 
## + Fold08.Rep4: alpha=0.36092, lambda=0.018533 
## - Fold08.Rep4: alpha=0.36092, lambda=0.018533 
## + Fold09.Rep4: alpha=0.30285, lambda=0.004666 
## - Fold09.Rep4: alpha=0.30285, lambda=0.004666 
## + Fold09.Rep4: alpha=0.02355, lambda=0.061703 
## - Fold09.Rep4: alpha=0.02355, lambda=0.061703 
## + Fold09.Rep4: alpha=0.32050, lambda=0.005575 
## - Fold09.Rep4: alpha=0.32050, lambda=0.005575 
## + Fold09.Rep4: alpha=0.03179, lambda=0.019563 
## - Fold09.Rep4: alpha=0.03179, lambda=0.019563 
## + Fold09.Rep4: alpha=0.55008, lambda=0.600447 
## - Fold09.Rep4: alpha=0.55008, lambda=0.600447 
## + Fold09.Rep4: alpha=0.71987, lambda=0.017336 
## - Fold09.Rep4: alpha=0.71987, lambda=0.017336 
## + Fold09.Rep4: alpha=0.16961, lambda=0.086625 
## - Fold09.Rep4: alpha=0.16961, lambda=0.086625 
## + Fold09.Rep4: alpha=0.61948, lambda=0.011854 
## - Fold09.Rep4: alpha=0.61948, lambda=0.011854 
## + Fold09.Rep4: alpha=0.18579, lambda=0.002510 
## - Fold09.Rep4: alpha=0.18579, lambda=0.002510 
## + Fold09.Rep4: alpha=0.36092, lambda=0.018533 
## - Fold09.Rep4: alpha=0.36092, lambda=0.018533 
## + Fold10.Rep4: alpha=0.30285, lambda=0.004666 
## - Fold10.Rep4: alpha=0.30285, lambda=0.004666 
## + Fold10.Rep4: alpha=0.02355, lambda=0.061703 
## - Fold10.Rep4: alpha=0.02355, lambda=0.061703 
## + Fold10.Rep4: alpha=0.32050, lambda=0.005575 
## - Fold10.Rep4: alpha=0.32050, lambda=0.005575 
## + Fold10.Rep4: alpha=0.03179, lambda=0.019563 
## - Fold10.Rep4: alpha=0.03179, lambda=0.019563 
## + Fold10.Rep4: alpha=0.55008, lambda=0.600447 
## - Fold10.Rep4: alpha=0.55008, lambda=0.600447 
## + Fold10.Rep4: alpha=0.71987, lambda=0.017336 
## - Fold10.Rep4: alpha=0.71987, lambda=0.017336 
## + Fold10.Rep4: alpha=0.16961, lambda=0.086625 
## - Fold10.Rep4: alpha=0.16961, lambda=0.086625 
## + Fold10.Rep4: alpha=0.61948, lambda=0.011854 
## - Fold10.Rep4: alpha=0.61948, lambda=0.011854 
## + Fold10.Rep4: alpha=0.18579, lambda=0.002510 
## - Fold10.Rep4: alpha=0.18579, lambda=0.002510 
## + Fold10.Rep4: alpha=0.36092, lambda=0.018533 
## - Fold10.Rep4: alpha=0.36092, lambda=0.018533 
## + Fold01.Rep5: alpha=0.30285, lambda=0.004666 
## - Fold01.Rep5: alpha=0.30285, lambda=0.004666 
## + Fold01.Rep5: alpha=0.02355, lambda=0.061703 
## - Fold01.Rep5: alpha=0.02355, lambda=0.061703 
## + Fold01.Rep5: alpha=0.32050, lambda=0.005575 
## - Fold01.Rep5: alpha=0.32050, lambda=0.005575 
## + Fold01.Rep5: alpha=0.03179, lambda=0.019563 
## - Fold01.Rep5: alpha=0.03179, lambda=0.019563 
## + Fold01.Rep5: alpha=0.55008, lambda=0.600447 
## - Fold01.Rep5: alpha=0.55008, lambda=0.600447 
## + Fold01.Rep5: alpha=0.71987, lambda=0.017336 
## - Fold01.Rep5: alpha=0.71987, lambda=0.017336 
## + Fold01.Rep5: alpha=0.16961, lambda=0.086625 
## - Fold01.Rep5: alpha=0.16961, lambda=0.086625 
## + Fold01.Rep5: alpha=0.61948, lambda=0.011854 
## - Fold01.Rep5: alpha=0.61948, lambda=0.011854 
## + Fold01.Rep5: alpha=0.18579, lambda=0.002510 
## - Fold01.Rep5: alpha=0.18579, lambda=0.002510 
## + Fold01.Rep5: alpha=0.36092, lambda=0.018533 
## - Fold01.Rep5: alpha=0.36092, lambda=0.018533 
## + Fold02.Rep5: alpha=0.30285, lambda=0.004666 
## - Fold02.Rep5: alpha=0.30285, lambda=0.004666 
## + Fold02.Rep5: alpha=0.02355, lambda=0.061703 
## - Fold02.Rep5: alpha=0.02355, lambda=0.061703 
## + Fold02.Rep5: alpha=0.32050, lambda=0.005575 
## - Fold02.Rep5: alpha=0.32050, lambda=0.005575 
## + Fold02.Rep5: alpha=0.03179, lambda=0.019563 
## - Fold02.Rep5: alpha=0.03179, lambda=0.019563 
## + Fold02.Rep5: alpha=0.55008, lambda=0.600447 
## - Fold02.Rep5: alpha=0.55008, lambda=0.600447 
## + Fold02.Rep5: alpha=0.71987, lambda=0.017336 
## - Fold02.Rep5: alpha=0.71987, lambda=0.017336 
## + Fold02.Rep5: alpha=0.16961, lambda=0.086625 
## - Fold02.Rep5: alpha=0.16961, lambda=0.086625 
## + Fold02.Rep5: alpha=0.61948, lambda=0.011854 
## - Fold02.Rep5: alpha=0.61948, lambda=0.011854 
## + Fold02.Rep5: alpha=0.18579, lambda=0.002510 
## - Fold02.Rep5: alpha=0.18579, lambda=0.002510 
## + Fold02.Rep5: alpha=0.36092, lambda=0.018533 
## - Fold02.Rep5: alpha=0.36092, lambda=0.018533 
## + Fold03.Rep5: alpha=0.30285, lambda=0.004666 
## - Fold03.Rep5: alpha=0.30285, lambda=0.004666 
## + Fold03.Rep5: alpha=0.02355, lambda=0.061703 
## - Fold03.Rep5: alpha=0.02355, lambda=0.061703 
## + Fold03.Rep5: alpha=0.32050, lambda=0.005575 
## - Fold03.Rep5: alpha=0.32050, lambda=0.005575 
## + Fold03.Rep5: alpha=0.03179, lambda=0.019563 
## - Fold03.Rep5: alpha=0.03179, lambda=0.019563 
## + Fold03.Rep5: alpha=0.55008, lambda=0.600447 
## - Fold03.Rep5: alpha=0.55008, lambda=0.600447 
## + Fold03.Rep5: alpha=0.71987, lambda=0.017336 
## - Fold03.Rep5: alpha=0.71987, lambda=0.017336 
## + Fold03.Rep5: alpha=0.16961, lambda=0.086625 
## - Fold03.Rep5: alpha=0.16961, lambda=0.086625 
## + Fold03.Rep5: alpha=0.61948, lambda=0.011854 
## - Fold03.Rep5: alpha=0.61948, lambda=0.011854 
## + Fold03.Rep5: alpha=0.18579, lambda=0.002510 
## - Fold03.Rep5: alpha=0.18579, lambda=0.002510 
## + Fold03.Rep5: alpha=0.36092, lambda=0.018533 
## - Fold03.Rep5: alpha=0.36092, lambda=0.018533 
## + Fold04.Rep5: alpha=0.30285, lambda=0.004666 
## - Fold04.Rep5: alpha=0.30285, lambda=0.004666 
## + Fold04.Rep5: alpha=0.02355, lambda=0.061703 
## - Fold04.Rep5: alpha=0.02355, lambda=0.061703 
## + Fold04.Rep5: alpha=0.32050, lambda=0.005575 
## - Fold04.Rep5: alpha=0.32050, lambda=0.005575 
## + Fold04.Rep5: alpha=0.03179, lambda=0.019563 
## - Fold04.Rep5: alpha=0.03179, lambda=0.019563 
## + Fold04.Rep5: alpha=0.55008, lambda=0.600447 
## - Fold04.Rep5: alpha=0.55008, lambda=0.600447 
## + Fold04.Rep5: alpha=0.71987, lambda=0.017336 
## - Fold04.Rep5: alpha=0.71987, lambda=0.017336 
## + Fold04.Rep5: alpha=0.16961, lambda=0.086625 
## - Fold04.Rep5: alpha=0.16961, lambda=0.086625 
## + Fold04.Rep5: alpha=0.61948, lambda=0.011854 
## - Fold04.Rep5: alpha=0.61948, lambda=0.011854 
## + Fold04.Rep5: alpha=0.18579, lambda=0.002510 
## - Fold04.Rep5: alpha=0.18579, lambda=0.002510 
## + Fold04.Rep5: alpha=0.36092, lambda=0.018533 
## - Fold04.Rep5: alpha=0.36092, lambda=0.018533 
## + Fold05.Rep5: alpha=0.30285, lambda=0.004666 
## - Fold05.Rep5: alpha=0.30285, lambda=0.004666 
## + Fold05.Rep5: alpha=0.02355, lambda=0.061703 
## - Fold05.Rep5: alpha=0.02355, lambda=0.061703 
## + Fold05.Rep5: alpha=0.32050, lambda=0.005575 
## - Fold05.Rep5: alpha=0.32050, lambda=0.005575 
## + Fold05.Rep5: alpha=0.03179, lambda=0.019563 
## - Fold05.Rep5: alpha=0.03179, lambda=0.019563 
## + Fold05.Rep5: alpha=0.55008, lambda=0.600447 
## - Fold05.Rep5: alpha=0.55008, lambda=0.600447 
## + Fold05.Rep5: alpha=0.71987, lambda=0.017336 
## - Fold05.Rep5: alpha=0.71987, lambda=0.017336 
## + Fold05.Rep5: alpha=0.16961, lambda=0.086625 
## - Fold05.Rep5: alpha=0.16961, lambda=0.086625 
## + Fold05.Rep5: alpha=0.61948, lambda=0.011854 
## - Fold05.Rep5: alpha=0.61948, lambda=0.011854 
## + Fold05.Rep5: alpha=0.18579, lambda=0.002510 
## - Fold05.Rep5: alpha=0.18579, lambda=0.002510 
## + Fold05.Rep5: alpha=0.36092, lambda=0.018533 
## - Fold05.Rep5: alpha=0.36092, lambda=0.018533 
## + Fold06.Rep5: alpha=0.30285, lambda=0.004666 
## - Fold06.Rep5: alpha=0.30285, lambda=0.004666 
## + Fold06.Rep5: alpha=0.02355, lambda=0.061703 
## - Fold06.Rep5: alpha=0.02355, lambda=0.061703 
## + Fold06.Rep5: alpha=0.32050, lambda=0.005575 
## - Fold06.Rep5: alpha=0.32050, lambda=0.005575 
## + Fold06.Rep5: alpha=0.03179, lambda=0.019563 
## - Fold06.Rep5: alpha=0.03179, lambda=0.019563 
## + Fold06.Rep5: alpha=0.55008, lambda=0.600447 
## - Fold06.Rep5: alpha=0.55008, lambda=0.600447 
## + Fold06.Rep5: alpha=0.71987, lambda=0.017336 
## - Fold06.Rep5: alpha=0.71987, lambda=0.017336 
## + Fold06.Rep5: alpha=0.16961, lambda=0.086625 
## - Fold06.Rep5: alpha=0.16961, lambda=0.086625 
## + Fold06.Rep5: alpha=0.61948, lambda=0.011854 
## - Fold06.Rep5: alpha=0.61948, lambda=0.011854 
## + Fold06.Rep5: alpha=0.18579, lambda=0.002510 
## - Fold06.Rep5: alpha=0.18579, lambda=0.002510 
## + Fold06.Rep5: alpha=0.36092, lambda=0.018533 
## - Fold06.Rep5: alpha=0.36092, lambda=0.018533 
## + Fold07.Rep5: alpha=0.30285, lambda=0.004666 
## - Fold07.Rep5: alpha=0.30285, lambda=0.004666 
## + Fold07.Rep5: alpha=0.02355, lambda=0.061703 
## - Fold07.Rep5: alpha=0.02355, lambda=0.061703 
## + Fold07.Rep5: alpha=0.32050, lambda=0.005575 
## - Fold07.Rep5: alpha=0.32050, lambda=0.005575 
## + Fold07.Rep5: alpha=0.03179, lambda=0.019563 
## - Fold07.Rep5: alpha=0.03179, lambda=0.019563 
## + Fold07.Rep5: alpha=0.55008, lambda=0.600447 
## - Fold07.Rep5: alpha=0.55008, lambda=0.600447 
## + Fold07.Rep5: alpha=0.71987, lambda=0.017336 
## - Fold07.Rep5: alpha=0.71987, lambda=0.017336 
## + Fold07.Rep5: alpha=0.16961, lambda=0.086625 
## - Fold07.Rep5: alpha=0.16961, lambda=0.086625 
## + Fold07.Rep5: alpha=0.61948, lambda=0.011854 
## - Fold07.Rep5: alpha=0.61948, lambda=0.011854 
## + Fold07.Rep5: alpha=0.18579, lambda=0.002510 
## - Fold07.Rep5: alpha=0.18579, lambda=0.002510 
## + Fold07.Rep5: alpha=0.36092, lambda=0.018533 
## - Fold07.Rep5: alpha=0.36092, lambda=0.018533 
## + Fold08.Rep5: alpha=0.30285, lambda=0.004666 
## - Fold08.Rep5: alpha=0.30285, lambda=0.004666 
## + Fold08.Rep5: alpha=0.02355, lambda=0.061703 
## - Fold08.Rep5: alpha=0.02355, lambda=0.061703 
## + Fold08.Rep5: alpha=0.32050, lambda=0.005575 
## - Fold08.Rep5: alpha=0.32050, lambda=0.005575 
## + Fold08.Rep5: alpha=0.03179, lambda=0.019563 
## - Fold08.Rep5: alpha=0.03179, lambda=0.019563 
## + Fold08.Rep5: alpha=0.55008, lambda=0.600447 
## - Fold08.Rep5: alpha=0.55008, lambda=0.600447 
## + Fold08.Rep5: alpha=0.71987, lambda=0.017336 
## - Fold08.Rep5: alpha=0.71987, lambda=0.017336 
## + Fold08.Rep5: alpha=0.16961, lambda=0.086625 
## - Fold08.Rep5: alpha=0.16961, lambda=0.086625 
## + Fold08.Rep5: alpha=0.61948, lambda=0.011854 
## - Fold08.Rep5: alpha=0.61948, lambda=0.011854 
## + Fold08.Rep5: alpha=0.18579, lambda=0.002510 
## - Fold08.Rep5: alpha=0.18579, lambda=0.002510 
## + Fold08.Rep5: alpha=0.36092, lambda=0.018533 
## - Fold08.Rep5: alpha=0.36092, lambda=0.018533 
## + Fold09.Rep5: alpha=0.30285, lambda=0.004666 
## - Fold09.Rep5: alpha=0.30285, lambda=0.004666 
## + Fold09.Rep5: alpha=0.02355, lambda=0.061703 
## - Fold09.Rep5: alpha=0.02355, lambda=0.061703 
## + Fold09.Rep5: alpha=0.32050, lambda=0.005575 
## - Fold09.Rep5: alpha=0.32050, lambda=0.005575 
## + Fold09.Rep5: alpha=0.03179, lambda=0.019563 
## - Fold09.Rep5: alpha=0.03179, lambda=0.019563 
## + Fold09.Rep5: alpha=0.55008, lambda=0.600447 
## - Fold09.Rep5: alpha=0.55008, lambda=0.600447 
## + Fold09.Rep5: alpha=0.71987, lambda=0.017336 
## - Fold09.Rep5: alpha=0.71987, lambda=0.017336 
## + Fold09.Rep5: alpha=0.16961, lambda=0.086625 
## - Fold09.Rep5: alpha=0.16961, lambda=0.086625 
## + Fold09.Rep5: alpha=0.61948, lambda=0.011854 
## - Fold09.Rep5: alpha=0.61948, lambda=0.011854 
## + Fold09.Rep5: alpha=0.18579, lambda=0.002510 
## - Fold09.Rep5: alpha=0.18579, lambda=0.002510 
## + Fold09.Rep5: alpha=0.36092, lambda=0.018533 
## - Fold09.Rep5: alpha=0.36092, lambda=0.018533 
## + Fold10.Rep5: alpha=0.30285, lambda=0.004666 
## - Fold10.Rep5: alpha=0.30285, lambda=0.004666 
## + Fold10.Rep5: alpha=0.02355, lambda=0.061703 
## - Fold10.Rep5: alpha=0.02355, lambda=0.061703 
## + Fold10.Rep5: alpha=0.32050, lambda=0.005575 
## - Fold10.Rep5: alpha=0.32050, lambda=0.005575 
## + Fold10.Rep5: alpha=0.03179, lambda=0.019563 
## - Fold10.Rep5: alpha=0.03179, lambda=0.019563 
## + Fold10.Rep5: alpha=0.55008, lambda=0.600447 
## - Fold10.Rep5: alpha=0.55008, lambda=0.600447 
## + Fold10.Rep5: alpha=0.71987, lambda=0.017336 
## - Fold10.Rep5: alpha=0.71987, lambda=0.017336 
## + Fold10.Rep5: alpha=0.16961, lambda=0.086625 
## - Fold10.Rep5: alpha=0.16961, lambda=0.086625 
## + Fold10.Rep5: alpha=0.61948, lambda=0.011854 
## - Fold10.Rep5: alpha=0.61948, lambda=0.011854 
## + Fold10.Rep5: alpha=0.18579, lambda=0.002510 
## - Fold10.Rep5: alpha=0.18579, lambda=0.002510 
## + Fold10.Rep5: alpha=0.36092, lambda=0.018533 
## - Fold10.Rep5: alpha=0.36092, lambda=0.018533 
## Aggregating results
## Selecting tuning parameters
## Fitting alpha = 0.361, lambda = 0.0185 on full training set

# Best tuning parameter
elastic_reg$bestTune

##       alpha     lambda
## 7 0.3609182 0.01853285

Metrics for elasticnet

# Make predictions on training set
predictions_train <- predict(elastic_reg, x)
eval_results(y_train, predictions_train, train) 

##        RMSE   Rsquare
## 1 0.6365787 0.3650057

# Make predictions on test set
predictions_test <- predict(elastic_reg, x_test)
eval_results(y_test, predictions_test, test)

##        RMSE   Rsquare
## 1 0.6693786 0.3435866

colnames(df)

##  [1] "fixed.acidity"        "volatile.acidity"     "citric.acid"         
##  [4] "residual.sugar"       "chlorides"            "free.sulfur.dioxide" 
##  [7] "total.sulfur.dioxide" "density"              "pH"                  
## [10] "sulphates"            "alcohol"              "quality"

ggplot(data = df, mapping = aes(x = total.sulfur.dioxide, y = quality)) + 
  geom_point()

ggplot(data = df, mapping = aes(x = residual.sugar, y = quality)) + 
  geom_point()

There is really no relationship. This is probably an exercise in clusterting or machine learning, but linear regression is insufficient.

ggplot(data = df, mapping = aes(x = alcohol, y = quality)) + 
  geom_point() +
  geom_smooth()

## `geom_smooth()` using method = 'gam' and formula 'y ~ s(x, bs = "cs")'